Vehicle automatic exterior light control

ABSTRACT

A system and method of automatically controlling vehicle headlamps including an image sensor and a controller to generate headlamp control signals.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a divisional of U.S. patent application Ser. No.11/231,232, filed on Sep. 20, 2005, which is a continuation of U.S.patent application Ser. No. 10/326,673, filed on Dec. 19, 2002, now U.S.Pat. No. 6,947,577, which is a continuation of U.S. patent applicationSer. No. 09/528,389, filed on Mar. 20, 2000, now U.S. Pat. No.6,611,610, which is a continuation-in-part of U.S. patent applicationSer. No. 09/151,487, filed on Sep. 11, 1998, now U.S. Pat. No.6,255,639, which is a continuation of U.S. patent application Ser. No.08/831,232, filed on Apr. 2, 1997, now U.S. Pat. No. 5,837,994. Priorityunder 35 U.S.C. §120 is hereby claimed on the above-identified patentapplications.

FIELD OF THE INVENTION

The present invention pertains to headlamp dimmers and components thatcan be used with a headlamp dimmer.

BACKGROUND OF THE INVENTION

Modern automotive vehicles include a variety of different lamps toprovide illumination under different operating conditions. Headlamps aretypically controlled to alternately generate low beams and high beams.Low beams provide less illumination, and are used at night to illuminatethe forward path when other vehicles are present. High beams outputsignificantly more light, and are used to illuminate the vehicle'sforward path when other vehicles are not present. Daytime running lightshave also begun to experience widespread acceptance.

Laws in various countries regulate vehicle illumination, and vehiclemanufacturers must build cars that comply with these regulations. Forexample, regulations set forth by the United States Department ofTransportation (DOT) regulate the light emissions of a vehicle's highbeam headlamps. Various state regulations are used to control the amountof glare experienced by drivers due to preceding vehicles (othervehicles traveling in the same direction) and oncoming vehicles(vehicles traveling in the opposite direction).

Known vehicle high beam headlamp emissions in accordance with the DOTregulations limit the intensity to 40,000 cd at 0°, 10,000 cd at 3°,3250 cd at 6°, 1500 cd at 9°, and 750 cd at 12°. An example of anemission pattern meeting this regulation is illustrated in FIG. 1 ofU.S. Pat. No. 5,837,994, entitled “CONTROL SYSTEM TO AUTOMATICALLY DIMVEHICLE HEAD LAMPS,” issued to Joseph S. Stam et al. on Nov. 17, 1998,the disclosure of which is incorporated herein by reference. In order toavoid an illuminance of 0.1 foot candles (fc) incident on anothervehicle at these angles, the vehicle high beam headlamps should bedimmed within 700 feet of another vehicle if the vehicles are at anangle of 0°, within 350 feet of another vehicle if the vehicles are at ahorizontal position of 3°, and 200 feet of the other vehicle if theposition of the other vehicle is at an angle of 6° to the longitudinalaxis of the controlled vehicle. It can thus be seen that a precedingvehicle directly in front of the controlled vehicle (i.e., at an angleof 0°) will need to be identified well prior to the controlled vehiclecatching up to the preceding vehicle, although the distance by which thecontrolled vehicle's headlamps must be dimmed for a preceding vehiclecan be somewhat less than for an oncoming vehicle because glare frombehind is usually less disruptive than oncoming glare.

In order to automatically control the vehicle headlamps, variousheadlamp dimmer control systems have been proposed. In order to preventdrivers of other vehicles from being subjected to excessive glarelevels, an automatic headlamp dimmer system must sense both theheadlights of oncoming vehicles as well as the taillights of precedingvehicles. Some systems that effectively detect headlights are unable toadequately detect taillights. Most prior systems are unable todistinguish nuisance light sources, such as reflectors, street signs,streetlights, house lights, or the like, from light sources that requireheadlight control. Accordingly, these systems are subject to undesirabledimming of the high beams when no other traffic is present and turningon the high beams when other vehicles are present. In addition to theundesirable performance, it is difficult for prior systems to complywith the legal requirements as described above for high beam controlwhile avoiding unnecessary dimming of the vehicle headlamps.

Fog lights are examples of other vehicle lights that are difficult tocontrol automatically. Vehicles are known to include forward andrearward directed fog lights. In Europe, it is known to provide a verybright red or white light on the back of the vehicle which isilluminated under foggy conditions. The fog lights must be turned ON assoon as the fog reduces visibility by a predetermined amount and mustturn OFF when the fog drops below that density. A reliable method ofautomatically controlling such fog lights has not been available.

Accordingly, there is a need for a more reliable and intelligentautomatic lamp control for a vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

The subject matter that is regarded as the invention is particularlypointed out and distinctly claimed in the claim portion that concludesthe specification. The invention, together with further objects andadvantages thereof, may best be understood by reference to the followingdescription taken in conjunction with the accompanying drawings, wherelike numerals represent like components, and in which:

FIG. 1 illustrates vehicles traveling on a common road.

FIGS. 2 a and 2 b illustrate an optical sensor system, FIG. 2 b showinga perspective view and FIG. 2 a showing a cross section of the opticalsensor system taken along plane 2 a-2 a in FIG. 2 b.

FIG. 3 is a plan view illustrating an image sensor used in the opticalsensor system according to FIGS. 2 a and 2 b.

FIG. 4 is a top plan view illustrating a lens structure used in theoptical sensor system according to FIGS. 2 a and 2 b.

FIG. 5 is a side elevation view illustrating the lens structureaccording to FIG. 4.

FIG. 6 is a graph illustrating wave transmissivity as a function oflight wavelength for the lens.

FIG. 7 shows a light sensitive surface of the image sensor andillustrates the regions of the image array which are impacted by thelight from each of the lenses.

FIG. 8 is a cross section illustrating another image sensor assemblytaken along the same plane as the assembly in FIG. 2 a.

FIG. 9 is a cross section illustrating yet another light sensor assemblytaken along the same plane as the assembly in FIG. 2 a.

FIG. 10 is a partial cross section of a rearview mirror assemblyillustrating an optical sensor system.

FIG. 11 is a circuit schematic illustrating a circuit for an opticalsensor system and an electrochromic mirror.

FIG. 12 is a circuit schematic illustrating a headlamp drive for thecircuit according to FIG. 11.

FIG. 13 is a circuit schematic illustrating a microcontroller circuitfor the circuit according to FIG. 11.

FIG. 14 is a flow chart illustrating operation of an electrochromicmirror and headlamp control.

FIG. 15 is a flow chart illustrating operations to acquire and analyzean image.

FIG. 16 is a flow chart illustrating operations to analyze an image andfind a light source.

FIG. 17 a is a flow chart illustrating a seed fill algorithm.

FIG. 17 b illustrates a pixel array impacted by a light source.

FIG. 18 is a flow chart illustrating operation to determine if lightsources are oncoming or preceding vehicles.

FIG. 19 is a state diagram illustrating the duty cycle associated withstates for a variable high beam lamp.

FIG. 20 illustrates operation rules for changes of state in FIG. 19.

FIG. 21 is a flow chart illustrating operation to provide speed varyingthresholds.

FIG. 22 is a chart illustrating the different regions of the imagearray.

FIG. 23 is a flow chart illustrating operation of the microcontroller toshift the regions in FIG. 22.

FIG. 24 is a front perspective view illustrating an image sensorassembly including an electronically alterable filter.

FIG. 25 is a side elevation view of the liquid crystal filter in theimage sensor assembly according to FIG. 24.

FIG. 26 a is an exploded perspective view illustrating an LED headlamp.

FIG. 26 b is a fragmentary cross section taken along plane 26 b-26 b inFIG. 26 a.

FIG. 26 c is a front plan view illustrating an alternate embodiment ofan LED lamp.

FIG. 26 d is a fragmentary cross section taken along plane 26 d-26 d inFIG. 26 c.

FIG. 27 is a top, front perspective view of an LED headlamp forprojecting light in more than one horizontal direction.

FIG. 28 is a top, front perspective view of an LED headlamp forprojecting light in more than one vertical direction.

FIGS. 29 a-29 d illustrates a method of manufacturing surface mountedfilters for an image sensor.

FIG. 30 is a chart illustrating the wavelengths passed by a red filtersurface mounted to an image sensor.

FIG. 31 is a chart illustrating the wavelengths passed by a cyan filtersurface mounted to an image sensor.

FIG. 32 illustrates another image sensor assembly.

FIG. 33 illustrates an electrical system including a wave sensitiveheadlamp control.

DETAILED DESCRIPTION OF THE INVENTION

A controlled vehicle 100 (FIG. 1) having an automatic headlamp dimmerincludes an optical sensor system 102 for detecting the headlamps 104 ofan oncoming vehicle 105 and the taillights 108 of a preceding vehicle110. The headlights 111 of the controlled vehicle 100 are controlledautomatically to avoid shining the high beams, or bright lights,directly into the eyes of a driver of oncoming vehicle 105 or byreflection into the eyes of the driver of the preceding vehicle 110. Theoptical sensor assembly 102 is illustrated mounted in the windshieldarea of the vehicle, but those skilled in the art will recognize thatthe sensor could be mounted at other locations that provide the sensorwith a view of the scene in front of the vehicle. One particularlyadvantageous mounting location is high on the vehicle windshield toprovide a clear view, which view can be achieved by mounting the opticalsensor assembly 102 in a rearview mirror mount, a vehicle headliner, avisor, or in an overhead console. Other views that may be advantageouslyemployed include mounting the optical sensor assembly 102 on theA-pilar, the dashboard, or at any other location providing a forwardviewing area. However, the most advantageous mounting locations arethose that position the image sensor to view a forward scene through anarea kept clean by the vehicle's windshield wipers.

With reference to FIGS. 2 a and 2 b, the optical sensor assembly 102includes an electronic image sensor 201 and an optical system to directlight onto the image sensor 201. The image sensor 201 generallycomprises an array of light sensitive components and associatedcircuitry to output electronic pixel light level signals responsive tolight impacting the surface of the image sensor 201. The optical systemgenerally contains four components: lens structure 202; aperture stopmember 203; far field baffle 204; and optional infrared filter 206. Theoptical system controls the scene viewed by the image sensor 201. Inparticular, the optical system focuses light rays 205 passing throughopening 207 of the far field baffle onto the array 201 contained withinthe image sensor assembly 201.

The Image Sensor

The configuration of the image sensor 201 is illustrated in FIG. 3. Theimage sensor includes an image array 301 (FIG. 3) that can be made fromany one of a variety of sensors, such as CMOS image sensors, chargecoupled device (CCD) image sensors, or any other suitable image sensor.In one embodiment, the image sensor is a CMOS photo gate active pixelimage sensor. A CMOS photo gate active pixel image sensor is describedin U.S. Pat. No. 5,471,515, entitled “ACTIVE PIXEL SENSOR WITHINTER-PIXEL CHARGE TRANSFER,” issued to Eric R. Fossum et al., on Nov.28, 1995, the disclosure of which is incorporated herein by referencethereto. Sensor systems including arrays are disclosed in U.S. patentapplication Ser. No. 09/448,364, entitled “CONTROL CIRCUIT FOR IMAGESENSORS,” filed Nov. 23, 1999, by Jon H. Bechtel et al., now U.S. Pat.No. 6,469,739; U.S. patent application Ser. No. 08/933,210, entitled“CONTROL CIRCUIT FOR IMAGE SENSORS,” filed on Sep. 16, 1997, by Jon H.Bechtel et al., now U.S. Pat. No. 5,990,469; and U.S. patent applicationSer. No. 08/831,232, filed Apr. 2, 1997, entitled “CONTROL SYSTEM TOAUTOMATICALLY DIM VEHICLE HEADLAMPS,” by Joseph S. Stam et al., now U.S.Pat. No. 5,837,994, the disclosures of these patents are incorporatedherein by reference thereto.

The array 301 may, for example, comprise photogate active pixels, suchas 10 to 50 μm pixels. It is advantageous for the array to be a lowresolution array, which is an array that has a resolution of less than7000 pixels per square millimeter and more preferably less than 2500pixels per square millimeter. The array may have 25 μm or larger photogate active pixels. In particular, the array may include 30 μm or largerpixels arranged in a grid smaller than 200 rows by 200 columns, and mayadvantageously comprise a rectangular array having 64 columns and 80rows of pixels. Such an image sensor is described in detail in U.S. Pat.No. 5,471,515, incorporated herein above by reference thereto. Theoptically active region of array 301 is approximately 1.9 mm in the Xdirection by 2.4 mm in the Y direction. Using such a low resolutionimage sensor array to monitor the forward field of the controlledvehicle 100 results in a relatively low resolution sensor system thatcan be supported by high speed processing in a cost effective mannerwhile enabling a vehicle headlight control to be highly reliable andaccurate. The low resolution array reduces the memory and processorrequirements of associated circuitry. The use of such larger pixelsincreases sensitivity and thus allows the use of slower, and thus lowercost, optics. However, one skilled in the art will recognize that theuse of a higher resolution array with smaller pixels may be advantageousto reduce the area of the sensor array and thus potentially reduce thecost of the array itself. In the case where a high resolution imagesensor is employed, faster components and higher quality optics may berequired to accommodate the high resolution image sensor. With advancesin optical manufacturing, such optics may become cost effective suchthat use of a higher resolution array becomes economically practical.Additionally, it is expected that in future years, the cost of aprocessor and memory required to process an image containing more pixelswill be reduced making a higher resolution system feasible. However, atthis time, it is preferred to use a low resolution array with relativelyfew pixels to maintain the economy of the invention and thus enablewidespread acceptance in the market place.

An image sensor and control circuit therefore will be described ingreater detail herein below, and such a system is disclosed in U.S. Pat.No. 5,990,469, the disclosure of which is incorporated herein above byreference.

In addition to the array 301, the image sensor 201 includes: serialtiming and control circuit 309; column output circuit 302; biasgenerating digital-to-analog converters (DACs) 305; flashanalog-to-digital converters (ADCs) 304; and analog amplifier 303.Serial timing and control circuitry 309 provides a high-speedsynchronous bi-directional interface between a microcontroller 1105(FIG. 11) and the image sensor 201. As described in U.S. Pat. No.5,990,469, the control circuit 309 allows the microcontroller 1105 tooutput parameters that control the selection of pixels exposed formeasurement (i.e., selects the area of array 301 exposed; which area isreferred to herein as the “window”); exposure time which affectssensitivity; bias voltages generated by the bias voltage generation DACs305; and the analog gain of amplifier 303. Additional features includethe ability to expose two windows simultaneously, using the same ordifferent gain settings for the amplifier 303 for pixels of therespective windows, and the ability to acquire a sequence of multipleframes. The control circuit 309 also enables a sleep feature thatdisables the analog components of the image sensor assembly 201 toreduce power consumption when the image sensor is not in use. The imagesensor also includes: a power supply input Vdd; a ground input; serialdata bus input/output (I/O) 308; serial data clock I/O 311; slave selectinput 307; and clock input 306.

The Lens Structure

Lens structure 202 (FIG. 2 a) includes lenses 208 and 209. Although twolenses are disclosed, the image sensor 201 could use a single lens ormore than two lenses. The two lenses 208,209 are used to produce twodifferent images of the same scene through different color filters toassist in properly discriminating headlamps from tail lamps using theimage sensor 201 as described below and in U.S. Pat. Nos. 5,837,994 and5,990,469, the disclosures of which are incorporated herein above byreference. The image system allows the acquisition of an image of ascene through at least one color filter 208, 209. In one embodiment, thelens structure is constructed to image the forward scene onto one regionof the image array 301 through a first color filter and to image theforward scene onto another region of the image array through a secondfilter. For example, filter 209 (FIGS. 2 a and 2 b) may be a red filterand filter 208 may be either a blue filter, a green filter, a cyanfilter, a clear filter (which is, for example, the absence of a colorfilter), or any other suitable filter. There are various places withinthe optical system that the filters could be incorporated other than thelens 208, 209, such as on the image sensor surface. However,incorporating the filters into the lens structure has the advantage thatthe light is not focused at the point of the filter. Locating the filterin the image plane, i.e., on the sensor, would leave an organic filtersusceptible to thermal damage should the sun fall within the sensor'sfield of view such that the sun's rays are focused on the filters.Typically, filter materials are much more vulnerable to thermal damagethan the image sensor itself. A possible exception would be dichroicinterference filters, which are highly resistant to thermal damage. Amethod by which such thermal resistant filters can be deposited onto asemiconductor image sensor surface is described in greater detail hereinbelow.

The lens structure 202 will now be described in greater detail withreference to FIGS. 2 a, 4, 5, and 7. The lens structure 202 includes afirst lens 208 and a second lens 209 that focus light from othervehicles onto the image array 301 (FIG. 7). The two lens elements 208,209 image the forward scene onto different respective regions 702, 703of the image sensor image array 301. Each lens element 208, 209 containsa spectral band pass filter, such that the forward image scene projectedonto the respective regions of the image array 301 each represents adifferent color component of the image, where it is advantageous todetermine the relative color of objects in the field of view. It isparticularly advantageous for the lens elements 208 (FIG. 2 a) and 209to comprise cyan and red filters, respectively, as mentioned above andshown in FIGS. 4 and 5. Alternatively, it may be advantageous for lens209 to contain a red spectral filter and for lens 208 to be clear.

The sensor system may, for example, employ lens elements 208 and 209that are 0.5 mm to 2.5 mm in diameter, and may advantageously compriselenses that are 1.0 to 2.5 mm in diameter, such as being 2.2 mm indiameter. In the Y direction, the lens center axes C₁ and C₂ may bespaced by 0.6 to 1.6 mm, and may, for example, be spaced by 1.1 to 1.4mm, and may advantageously be spaced 1.3 mm in the Y direction (asindicated in FIG. 5). A portion of the lens may be truncated on one sideto achieve this spacing. The lens center C₁, C₂ may be aligned in the Xdirection as shown in FIG. 4, or be offset. Lens element 208 may, forexample, include a cyan filter with an aspheric lens having a curvatureof 0.446 mm⁻¹ and a conic constant of −0.5 to achieve the desired focallength. Lens element 209 may, in contrast, be a red filter aspheric lenswith a curvature of 0.450 mm⁻¹ and a conic constant of −0.5 to achievethe same desired focal length. At the center, each lens is 0.5 to 1.5 mmthick, and may advantageously be 1.0 mm thick. The difference incurvature of the two lenses compensates for the dispersion in the lensmaterial and optimizes each lens for the spectral band passed by thefilter. These parameters result in a lens with an effective focal lengthof 4.5 mm and thus have an F# of 2. It will be recognized that theseoptics are exemplary, and that other optics could be provided, such thatthe focal length and F# could be different.

The lens structure 202 may be molded out of a light-transmissive plasticsuch as acrylic, manufactured from glass, or may be produced by anyother suitable construction. Where the lenses 208, 209 are molded fromplastic, the filters may be formed integrally with the plastic by theinclusion of dyes within the material or they may be molded of a cleartransparent material, particularly where the image sensor has a surfacemounted filter as described herein below. An example of an acrylicmaterial, which includes a red filter dye, is part number RD-130available form OptiColor, Inc. of Huntington Beach, Calif. USA. Anacrylic material incorporating a cyan filter is OptiColor part numberBL-152. The spectral transmission of these two materials is shown inFIG. 6. Using the optional infrared filter 206 will remove light aboveapproximately 700 nm.

The lens structure 202 including integral filters may be manufacturedusing a bi-color injection molding process. First one-half of the lens,for example the red half including lens 209, is molded in a toolcontaining features to form the red half of the lens. Next, the moldedred half of the lens is inserted into a tool containing features for thecyan half of the lens structure 202 in addition to features to hold thered lens 209. The cyan lens 208 half of the lens structure 202 is theninjection molded against the red lens 209 half forming one bi-colorlens. Alternatively, each of the lenses 208 and 209 can be provided bylens elements such as disclosed in copending U.S. Pat. No. 6,130,421,entitled “IMAGING SYSTEM FOR VEHICLE HEADLAMP CONTROL,” filed Jun. 9,1998, by Jon H. Bechtel et al., the disclosure of which is incorporatedherein by reference thereto.

While red and cyan filtered lens elements are used in the illustratedembodiment, other combinations of lenses may also be suitable for thisapplication. For example, it is possible to replace cyan lens element208 with a clear lens. Also, it is possible to use three lenses withrespective color filters and, in particular, red, green, and bluefilters, to obtain a full color image. Such an arrangement could use asequence of three lenses aligned along the Y axis of FIG. 4, with onelens positioned on the center axis and the other two lenses positionedadjacent this center lens. The spacing of the lenses mightadvantageously be uniform to provide uniformly spaced regions on thelight sensitive surface of the image array 301. The red and cyan filtercolors described above are thus only presented herein as an example, andany combination of filters which pass at least two isolated oroverlapping spectral bands of light and allow for the distinction oftail lamps from headlamps may be used. Those skilled in the art willrecognize that other methods can be used to incorporate the filters,such as screen printing dyes applied to the flat back surface of thelens structure 202, or application of a filter material to the surfaceof a clear lens structure. Additionally, an advantageous system using asingle lens is described herein below with reference to FIGS. 24 and 25.

The Aperture Stop

Aperture stop 203 comprises an opaque member, including apertures 240(FIG. 2 a) and 242, positioned over lenses 208, 209. The aperture stop203 can be manufactured of any suitable material, such as moldedplastic, and it can be painted or otherwise treated so as to block thepassage of light if the material of which the plastic is manufactured isnot opaque. Aperture stop 203 defines the apertures 240, 242 for lenselements 208 and 209. Aperture stop 203 also prevents passage of straylight through regions of lens structure 202 other than the lens elements208 and 209. It will be recognized that the aperture stop 203 can bepaint applied directly to the surface of lens structure 202, andoptionally to the sidewalls of light sensor assembly 250, such that thepaint blocks passage of stray light through regions of lens 202 otherthan the lens elements 208 and 209.

The Far Field Baffle

The far field baffle 204 (FIGS. 2 a and 2 b) is an opaque enclosure tobe positioned over the image sensor 201. The baffle includes an opening207, which is the sole light passage into the image sensor. Theillustrated far field baffle 204 is a generally rectangular boxincluding four sidewalls 215 (only two of the four being visible in FIG.2 b), an end wall 217 including opening 207, and an open end 219. Theopen end is secured to the support 220 on which image sensor 201 iscarried. The support 220 may for example be a circuit board, or ahousing, and is preferably opaque to block the passage of light into thechamber defined by the far field baffle. The walls 215, 217 of the farfield baffle 204 are opaque, and may be of any suitable constructionsuch as stamped from metal, molded from plastic, or the like. If thematerial from which the walls are made is not opaque, it may be paintedor otherwise treated to block the admission of light. The far fieldbaffle defines the forward scene viewed by image sensor array 208. Theside walls 215 and end wall 217 prevent light at angles outside of thedesired field of view from entering and are also used to keep lightinput through one lens from crossing over to the region of the arrayreserved for the other lens. The far field baffle aperture 207 isideally about 4-6 focal lengths, or approximately 18 mm in theillustrated embodiment, from the front of the lens (for the sake ofclarity, the figures of the application are not to scale). The field ofview through aperture 207, aperture stop 203, and lenses 208 and 209, inthe illustrated embodiment, is about 10° in the vertical direction and25° in the horizontal direction in front of the vehicle. This field ofview can be achieved with a rectangular or elliptical far field baffleopening 207 that is 6 to 7 mm in the Y direction and 9 to 10 mm in the Xdirection, in the above-described embodiment.

In particular, the far field baffle 204 has an opening 207 in an endwall 217. The sidewalls 215 of the image array sensor extendorthogonally from the end wall 217. The walls 215, 217 may be formedintegrally in a molding or stamping process or they may be joined afterconstruction using an adhesive, fasteners or the like. The far fieldbaffle is preferably a black plastic molded member, although it may beprovided using any material that will absorb most or all of the lightstriking the sidewalls. By providing wall surfaces on the inside of thefar field baffle that absorb light, the walls will not reflect lightthat enters though opening 207 onto the image array sensor 201. In theillustrated embodiment, the baffle is rectangular, but those skilled inthe art will recognize that the baffle could be square, cylindrical, orany other suitable shape. An imaging system including a far field baffleis described in U.S. Pat. No. 6,130,421, entitled “IMAGING SYSTEM FORVEHICLE HEADLAMP CONTROL,” filed on Jun. 9, 1998, by Jon H. Bechtel etal, the disclosure of which is incorporated herein by reference thereto.

Infrared Filter

The far field baffle holds an optional infrared filter 206 (FIG. 2 a).Infrared filter 206 prevents light of wavelengths longer than about 700nm from being imaged by the optical system. This is advantageous aslight above 700 nm (FIG. 6) will pass through the red and cyan filters.By removing this light, the only light that will be considered isvisible light in the pass band of the red and blue filters. Infraredfilters are available from Optical Coating Laboratories of Santa Rosa,Calif. and are called “Wide Band Hot Mirrors.” The infrared filter 206may be mounted to the end wall 217 using an adhesive, mechanicalfasteners such as a snap connector, or the like, and may seal off thechamber within the far field baffle to prevent dust and moisture fromentering the system and degrading the performance of the system.

Alternatively, infrared filter 206 may be incorporated as a dye withinthe lens, a coating on the lens, a coating on the image sensor surface,a lid on an image sensor package, or elsewhere in the image sensorassembly. If the IR filter is not such that it can be used to close theopening 217 of far field baffle 204, it may be desirable to place aclear window, such as glass, plastic, or the like in the opening 217 toprevent dust from entering into the interior of the far field baffle andinterfering with the performance of the sensor system 102.

Assembly of the Image Sensor Assembly

Assembly of the image sensor will now be described with reference toimage sensor assembly 801 of FIG. 8, which is identical to the imagesensor assembly 250 except for the gel 805 in image sensor assembly 801.The image sensor 201, lens structure 202, and aperture stop 203 arecombined to form an integral image sensor assembly 250 (FIG. 2) or 801(FIG. 8). The image sensor 201, which is advantageously a singleintegrated circuit (IC), is attached to printed circuit board 220 by anysuitable conventional means such as using chip-on-board technology.Connections to the image sensor chip are made by any suitable means,such as wire bonds 804. The bonded IC is then optionally covered with anoptically clear stress relieving gel 805. Examples of materials that canbe used for this coating are Silicon Semi-Gel type C from Transene Co.of Danvers, Mass. or Dielectric Gel 3-6211 from Dow Corning Corp. ofMidland, Mich. The coated IC is then encapsulated in a hard opticallyclear enclosure 802, which may, for example, comprise epoxy. The epoxyis formed into a desired shape, and may, for example, form a cube. Thecube can be dimensioned to occupy a very small volume, and may havelength and width dimensions of about 1 cm on a side, and a thickness ofabout 5 mm. The enclosure 802 (FIG. 8) may be selected to haveapproximately the same index of refraction as the stress relieving gel805 to prevent any refraction at the interface between these twomaterials. Examples of suitable epoxies are: Epo-Tek 301-2FL from EpoxyTechnology, Inc. of Billerica, Mass. or Epoxy 50 from Transene Co, orDexter-Hysol OS 1900. If the coefficient of thermal expansion of theenclosure 802 is sufficiently low that its expansion and contractionwill not break wire bonds 804 at the expected operating temperaturerange for the image array sensor 201, stress-relieving gel 805 can beomitted, as is shown by enclosure 230 over wire bonds 234 in imagesensor assembly 250 in FIG. 2 a.

Lens structure 202 is attached to the enclosure 802 (FIG. 8) orenclosure 230 (FIG. 2) using a UV curable optically clear adhesive 232.UV curable adhesive 232 is dispensed onto the epoxy cube 802 and lens202 is juxtaposed with UV curable adhesive 232. Lens structure 202 isspaced from the image sensor 201 by a distance such that images at“infinity” are focused on the desired image regions 702 and 703. The UVcurable adhesive 232 is exposed to UV light and cured, locking lensstructure 202 into position and permanently attaching lens structure 202to enclosure 802. The total distance between the back surface of thelens structure 202 and the top of the image sensor 201 die is 6.7 mm inthe illustrated example. This distance is significantly longer than theeffective focal length of 4.4 mm because the entire optical path betweenthe back of the lens 202 and the front of the image array 201 is througha material with a higher index of refraction than air. Ideally, theprocess of aligning the lens to the image sensor and curing the UVadhesive to hold it in place is accomplished while actively focusing thelens to accommodate variations in the manufacture of the lens and otherimage sensor assembly components. This process is accomplished bypowering the image sensor during assembly and acquiring images of a farfield scene from the image sensor into a host computer. The UV curableadhesive is dispensed onto the surface of the sensor and the lens ispositioned on the UV curable adhesive using a multi-axis robot orpositioner. The position of the lens is adjusted by the robot until theimages acquired by the sensor appear in focus. At this point, the UVcurable adhesive is exposed to UV light, cementing the lens into place.

UV curable adhesive 232 serves to fill the space between the lens 202and enclosure 802, and thus fills in any ripples or other non-planarsurfaces of enclosure 802 thereby precluding the creation of air gapsbetween the lens 202 and the enclosure 802. To accomplish the desirableoptical characteristics described hereinabove, UV curable adhesive 232should have approximately the same index of refraction as enclosure 802.This structure has the distinct advantage of minimizing the number ofoptical surfaces wherein a significant mismatch of indices of refractionbetween two different mediums can occur, thus increasing the opticalefficiency of the imaging system and reducing stray light. A suitableoptically clear UV cured adhesive is Norland Optical Adhesive 68manufactured by Norland Products, Inc., of New Brunswick, N.J. Othermaterials suitable for making the image sensor assembly 801 areavailable from Dymax.

The block 802 is completed by attaching the aperture stop 203 to lens202. If the aperture stop is a member, it may be attached to the outersurface of the lens using an adhesive, one or more mechanical fasteners,or the like. If the aperture stop is paint, it may be applied directlyto the surface of the lens element 202 after the lenses 208 and 209 arecovered with a removable mask, such as tape. After the paint dries, themask can be removed. The optical assembly 801 is then mounted to asupport 220. In particular, the image sensor array 201 is physicallymounted on a base substrate 221 by conventional means, and electricallyconnected to circuitry (not shown in FIG. 2 b) by electrical connectorssuch as wire bonds, solder, one or more connectors, or the like. Thebase substrate 221 may, for example, be a printed circuit board.

The support 220 may be constructed of the same material as the far fieldbaffle 204, or it may be constructed of a different material. The basesubstrate 221 may be omitted if the far field baffle and the imagesensor are mounted directly to either the support 220 or the housing(not shown) that carries the optical sensor assembly. For example, thesupport 220 may be a printed circuit board to which the image sensor 201and the far field baffle are connected. Regardless of whether the basesubstrate is provided, the far field baffle is mounted to the support220 or the housing (not shown) using an adhesive, a snap connector, amechanical fastener, or the like.

An image sensor assembly 901 according to an alternate embodiment isillustrated in FIG. 9. In this embodiment, the image sensor 201 ispackaged using more conventional electronic packaging, such as a ceramicpackage with a glass lid or a clear plastic package, which may, forexample, be a quad flat pack or a dual-in-line (DIP) package. Thepackaged image sensor 901 is mounted by suitable conventional means suchas by soldering, to printed circuit board 902. An ultraviolet (UV)curable adhesive 905 is then dispensed onto the packaged image sensor901. The adhesive used can be the same adhesive described above withrespect to adhesive 232. The thickness of the UV curable adhesive isdependent on the packaged image sensor 901 type. If the requiredthickness is too great, layers of the UV curable adhesive can be builtup or another material, such as an epoxy layer, can be sandwichedbetween the UV curable adhesives to decrease the thickness of theadhesive layer 905. The epoxy may be the same material described abovewith respect to the enclosure 230, 802. The lens structure 202 isjuxtaposed with the UV curable adhesive 905 and focused in the mannerpreviously described. Finally, the aperture stop 203 is attached to thelens structure 202 using an adhesive (not shown), mechanical fastener,or the like.

In addition to the means described herein above, the lens structure 202may be supported relative to the image sensor by other means, such as amechanical support. Such a structure is disclosed in U.S. Pat. No.6,130,421, entitled “IMAGING SYSTEM FOR VEHICLE HEADLAMP CONTROL,” filedon Jun. 9, 1998, the disclosure of which is incorporated herein byreference thereto. The same may also be used to position and maintainthe relative relationship between the components of the optical assemblyincluding the aperture stop and the far field baffle. A mechanicalfastening arrangement is disclosed herein below with respect to FIG. 24.

Mirror Mounted Image Sensor Assembly

As mentioned above, the headlamp dimmer can be advantageously integratedinto a rearview mirror 1000 as illustrated in FIG. 10, wherein the lightsensor assembly 201 is integrated into an automatic dimmingelectrochromic (EC) mirror subassembly 1001, or other variablereflectance mirror assembly. This location provides an unobstructedforward view through a region of the windshield of the vehicle that istypically cleaned by the vehicle's windshield wipers (not shown).Additionally, mounting the image sensor in the mirror assembly permitssharing of circuitry such as the power supply, microcontroller, andlight sensors. More specifically, the same ambient light sensor may beused to provide an ambient light measurement for both the auto-dimmingmirror function and the headlamp control function.

Referring to FIG. 10, light sensor assembly 801 is mounted within arearview mirror mount 1003, which is mounted to the vehicle windshield1002. The rearview mirror mount 1003 provides an opaque enclosure forthe image sensor. The infrared filter 206 can be mounted over a hole1007 in the rearview mirror mount 1003, as is shown. Alternatively, thefar field baffle 214 can be used with the infrared filter 206 mountedtherein. If the far field baffle 214 is used, it is mounted to thecircuit board 1008 with the image sensor assembly 202. Regardless ofwhether the far field baffle is used, the circuit board 1008 is mountedto rear view mirror mount 1003 using mounting brackets 1020 and 1021.The mounting brackets may be implemented using any suitableconstruction, such as metal brackets, plastic brackets which can beformed either integrally with the housing 1003 or as separatecomponents, mechanical fasteners which engage the circuit board 1008, orthe like. The separate brackets can be attached using an adhesive, metalfasteners, or other mechanical fastening means. Image sensor assembly201 is thus attached to, and held stationary by, the rear view mirrormount 1003 which is securely attached to the vehicle windshield or roofby conventional means.

A connector 1005 is connected to circuit board 1008 using a suitablecommercially available circuit board connector (not shown), which inturn is connected to the image sensor 201 through circuit board 1008.The connector 1005 is connected to a main circuit 1015 through a cable1006. The main circuit board is mounted within rearview mirror housing1004 by conventional means. Power and a communication link with thevehicle electrical system, including the headlamps 111 (FIG. 1), areprovided via a vehicle wiring harness 1017 (FIG. 10).

The Electrical System

The image sensor 201 electrically connected to the main circuit board1015 and mounted in the vehicle rearview mirror housing 1004 (FIG. 10)is represented in FIG. 11. The microcontroller 1105 receives imagesignals from the image sensor 201, processes the images, and generatesoutput signals. Although described with reference to a circuit boardmounted in a rearview mirror housing, the circuit board 1105 can bemounted in a vehicle accessory, such as a sun visor, overhead console,center console, dashboard, prismatic rearview mirror, A-pillar, or atany other suitable location in the vehicle. Should the controlledvehicle (100 in FIG. 1) include an electrochromic mirror, the circuitryfor the electrochromic mirror preferably shares the circuit board 1015(FIG. 10) with microprocessor 1105. Thus, the main circuit board 1015 ismounted within the mirror housing 1004. The EC circuitry furtherincludes ambient light sensor 1107 and glare light sensor 1109, whichmay advantageously be digital photodiode light sensors as described inU.S. patent application Ser. No. 09/491,192 entitled “PHOTODIODE LIGHTSENSOR,” filed Jan. 25, 2000, now U.S. Pat. No. 6,379,013, and U.S.patent application Ser. No. 09/307,191 entitled “VEHICLE EQUIPMENTCONTROL WITH SEMICONDUCTOR LIGHT SENSORS,” filed May 7, 1999, now U.S.Pat. No. 6,359,274, the disclosures of which are incorporated herein byreference. Microcontroller 1105 uses inputs from ambient light sensor1107 and glare lights sensor 1109 to determine the appropriate state forthe electrochromic mirror element 1102. The mirror is driven by ECmirror drive circuitry 1111, which may be a drive circuit described inU.S. Pat. No. 5,956,012, entitled “SERIES DRIVE CIRCUIT,” filed byRobert R. Tumbull et al. on Sep. 16, 1997, and PCT Application SerialNo. PCT/US97/16946, entitled “INDIVIDUAL MIRROR CONTROL SYSTEM,” filedby Robert C. Knapp et al. on Sep. 16, 1997; and U.S. patent applicationSer. No. 09/236,969, entitled “AUTOMATIC DIMMING MIRROR USINGSEMICONDUCTOR LIGHT SENSOR WITH INTEGRAL CHARGE COLLECTION”, filed May7, 1999, by Jon H. Bechtel et al., now abandoned, the disclosures ofwhich are incorporated herein by reference thereto. Other drivercircuits are known that can be used to drive the EC element 1102. The ECmirror drive circuit 1111 provides current to the EC element 1102through signal output 1127.

The microcontroller 1105 can take advantage of the availability ofsignals (such as vehicle speed) communicated over the vehicle'selectrical bus in making decisions regarding the operation of theheadlamps 111, which are represented by high beams 1131 and low beams1132 in FIG. 11, and the electrochromic mirror 1102. In particular,speed input 1117 provides vehicle speed information to themicrocontroller 1105, from which vehicle speed criteria can be used fordetermining the control state for the headlamps 111. The reverse signal1119 informs microcontroller 1105 that the vehicle is in reverse,responsive to which the microcontroller 1105 clears the electrochromicmirror element 1102 regardless of the signals output from the lightsensors 1107, 1109. Auto ON/OFF switch input 1121 is connected to aswitch having two states to dictate to microcontroller 1105 whether thevehicle headlamps 1131, 1132 should be automatically or manuallycontrolled. The auto ON/OFF switch (not shown) connected to the ON/OFFswitch input 1121 may be incorporated with the headlamp switches thatare traditionally mounted on the vehicle dashboard or incorporated intosteering wheel column levers. Manual dimmer switch input 1123 isconnected to a manually actuated switch (not shown) provides a manualoverride signal for the high beam state. Should the current controlledstate of the high beams be ON, the microcontroller will respond toactuation signal manual override signal control input 1123 to turn thehigh beams OFF temporarily until the driver restores operation or,optionally, until a predetermined time has elapsed. Alternatively,should the high beams be OFF, the microcontroller 1105 will respond toan actuation signal on input 1123 to turn the high beams ON. The manualhigh beam control switch can be implemented using a lever switch locatedon the steering column of controlled vehicle 100 (FIG. 1).

The circuit board 1101 has several outputs. The control signal onelectrochromic output 1127 provides current to the electrochromicelement 1102. Additional outputs (not shown) may optionally be providedto control exterior electrochromic rearview mirrors (not shown) if suchadditional mirrors are provided. The microcontroller 1105 communicatesthe current state of the low beam headlamps 1131 and the high beamheadlamps 1132 to the headlamp drive 1104 via headlamp control output1127. The microcontroller 1105 generates control signals communicatedover conductor 1113 (FIG. 11) to an optional visual indicator 1115 whichdisplays the current state of the high beam headlamps to the driver ofcontrolled vehicle 100. The high beam indicator is traditionally locatedin or near the vehicle's instrument cluster on the vehicle dashboard. Acompass sensor 1135 may be connected to the circuit board 1015 via abi-directional data bus 1137. The compass can be implemented using acommercially available compass of the type generating digital or analogsignals indicative of the vehicle's heading, such as those described inU.S. Pat. No. 5,239,264 entitled “ZERO-OFFSET MAGNETOMETER HAVING COILAND CORE SENSOR CONTROLLING PERIOD OF AN OSCILLATOR CIRCUIT”; U.S. Pat.No. 4,851,775 entitled “DIGITAL COMPASS AND MAGNETOMETER HAVING A SENSORCOIL WOUND ON A HIGH PERMEABILITY ISOTROPIC CORE”; U.S. Pat. No.5,878,370 entitled “VEHICLE COMPASS SYSTEM WITH VARIABLE RESOLUTION”;U.S. Pat. No. 5,761,094, entitled “VEHICLE COMPASS SYSTEM”; U.S. Pat.No. 5,664,335, entitled “VEHICLE COMPASS CIRCUIT”; U.S. Pat. No.4,953,305 entitled “VEHICLE COMPASS WITH AUTOMATIC CONTINUOUSCALIBRATION”; U.S. Pat. No. 4,677,381 entitled “FLUX-GATE SENSORELECTRICAL DRIVE METHOD AND CIRCUIT”; U.S. Pat. No. 4,546,551 entitled“ELECTRICAL CONTROL SYSTEM”; U.S. Pat. No. 4,425,717 entitled “VEHICLEMAGNETIC SENSOR”; and U.S. Pat. No. 4,424,631 entitled “ELECTRICALCOMPASS”, the disclosures of all of these patents are herebyincorporated herein by reference. A fog lamp control 1141 can beconnected to receive via fog light control output 1142 control signalsgenerated by microcontroller 1105. Fog lamp control 1141 controls frontfog lamps 1143 and rear fog light 1145 to turn ON and OFF.

Some or all of the inputs 1117, 1119, 1121, 1123, and 1135, and outputs1127, 1113, 1127, and 1142, as well as any other possible inputs oroutputs, can optionally be provided through a vehicle communications bus1125 shown in FIG. 11. Vehicle bus 1125 may be implemented using anysuitable standard communication bus, such as a Controller Area Network(CAN) bus. If vehicle bus 1125 is used, microcontroller 1105 may includea bus controller or the control interface may be provided by additionalcomponents on the main control board 1015.

FIG. 12 illustrates a headlamp drive 1104 including a drive circuit 1203for low beam headlamps 1131 and a drive circuit 1201 for high beamheadlamps 1132. Bus 1127 includes respective wires 1206 and 1207carrying pulse width modulated (PWM) signals generated bymicrocontroller 1105 for driving low beam headlamps 1131 and high beamheadlamps 1132. Alternatively, headlamp drive 1104 may contain a DCpower supply to vary the voltage supplied to the lamps 1131, 1132, andthus their brightness, in response to control signals on output 1127.Yet another alternative envisioned is to vary the aim of the high beamheadlamps 1131 as is described hereinbelow and as taught in U.S. Pat.No. 6,049,171, entitled “CONTINUOUSLY VARIABLE HEADLAMP CONTROL,” filedby Joseph S. Stam et al. on Sep. 18, 1998, the disclosure of which isincorporated herein by reference.

Headlamp drive 1104 provides power to the high beam 1131 and low beam1132 headlamps. In the simplest case, the headlamp drive contains relaysengaged in response to signal 1127 to turn ON or OFF the headlamps. In amore preferred embodiment, low and high beam headlamps 1131 and 1132fade ON or OFF under the control of headlamp drive 1104, which generatesa variable control signal. Such a control system is described incopending U.S. Pat. No. 6,049,171, incorporated herein above byreference thereto. In general, the patent teaches variable emissioncontrol of the headlamps can be provided by energizing the headlampsusing a pulse width modulation (PWM) supply, wherein the duty cycle ofthe drive varied between 0% and 100% to effect a continuously variablebrightness from the headlamps 1131, 1132.

The microcontroller 1105 analyzes images acquired by the image sensorassembly 201 responsive to which it detects oncoming or precedingvehicles in the forward field of view. The microcontroller 1105 usesthis information in conjunction with the various other inputs thereto todetermine the current control state for the headlamps 1131, 1132. Thecurrent control state of the headlamps refers to the brightness of thehigh beams and the low beams. In a variable control system, thisbrightness is varied by changing the duty cycle of the beams or the DCvoltage applied to the lamps as described above. In a non-variablesystem, the control state refers to whether the high beams and low beamsare ON or OFF.

A more detailed schematic showing the connections to the microcontroller1105 is shown in FIG. 13. The microcontroller 1105 can be implementedusing a microcontroller, a microprocessor, a digital signal processor, aprogrammable logic unit, discrete circuitry or combination thereof.Additionally, the microcontroller may be implemented using more than onemicroprocessor.

Operation

The combined effect of the lens structure 202 and the far field baffle204 will first be described with respect to FIGS. 2 a and 7. Asmentioned above, in one embodiment, the image array 301 contains 64columns and 80 rows of 30 μm pixels. The forward scene imaged throughthe red lens 209 is located on one region 703 of the image array 301.The forward scene imaged through the other lens element 208 is locatedon region 702 of the image array 301. In the embodiment illustrated,each of these regions is a 60 wide by 20 high pixel subwindow of theimage array. The centers of the two regions 702 and 703 are separated1.2 mm in the Y direction, the same spacing as the center axis of lens208 and 209. Fourteen pixel rows 704 define a band that lies between thetwo regions 702, 703 and serves as a border, or buffer, separating thesetwo regions.

The image sensor assembly 250 provides several advantages. Because theblock will be solid, it eliminates any surfaces between the image sensor201 die and the lens structure 202. By eliminating these surfaces, straylight is reduced. Second, the preferred embodiment allows for the activealignment of the lens, which allows for compensation of variousmanufacturing variances, as is described in greater detail herein above.Finally, the assembly is inexpensive, eliminating the need for costlyceramic packaging. Some or all of the above-mentioned advantages can berealized through variations on this structure. For example, theenclosure 230, UV curable adhesive 232, and possibly the stressrelieving gel 805 (FIG. 8) can be replaced with a UV cured epoxyadhesive.

In operation, an image is exposed onto the image array 301 (FIG. 3) foran exposure period, which may also be referred to herein as anintegration period. At the end of the exposure period, an output signalis stored for each of the pixels, and preferably is stored in the pixelsas is the case with the photogate pixel architecture described in U.S.Pat. No. 5,471,515 previously incorporated herein by reference. Theoutput signal from each of the pixels is representative of theillumination sensed by each pixel. This output signal is transferred tothe column output circuitry 302 one row at a time. The column outputcircuitry includes capacitors storing the respective pixel outputsignals for each pixel in the row. Next, the pixel output signals aresuccessively amplified by an analog amplifier 303. The amplifier gain isadvantageously adjustable, and may, for example, be controlled toselectively increase the amplitude of the amplifier input signal by 1(unity gain) to 15 times, in integer increments. Adjustment of the gainof the amplifier permits adjustment of the system sensitivity. Theamplified analog signals output from amplifier 303 are sampled by aflash analog-to-digital converter (ADC) 404. The flash ADC 404 convertseach of the amplified analog signals, which correspond to respectivepixels, into eight-bit digital grey scale values. Various bias voltagesfor the sensor are generated by digital-to-analog converter (DAC) 105.Two of the voltages generated by the bias generators are the ADC highand low reference values, which determine the analog voltages that willcorrespond to digital values of 255 and 0, respectively, thus settingthe range of the ADC.

Serial timing and control circuitry 309 dictates the timing sequence ofthe sensor operation, and is described in detail in co-pending U.S. Pat.No. 5,990,469, entitled “CONTROL CIRCUIT FOR IMAGE ARRAY SENSORS,”issued to Jon H. Bechtel et al on Nov. 23, 1999, the disclosure of whichis incorporated herein by reference.

The image sensor control process will now be described beginning withreference to FIG. 14. The control process may include control of anelectrochromic (EC) mirror. However, because processes for controllingan EC mirror are well known, such processes are not completely describedherein. Electrochromic devices are generally known, and examples ofelectrochromic devices and associated circuitry, some of which arecommercially available, are disclosed in Byker U.S. Pat. No. 4,902,108;Bechtel et al. Canadian Patent No. 1,300,945; Bechtel U.S. Pat. No.5,204,778; Byker Pat. No. 5,280,380; Byker Pat. No. 5,336,448; Bauer etal. Pat. No. 5,434,407; Tonar Pat. No. 5,448,397; Knapp Pat. No.5,504,478; Tonar et al. Pat. No. 5,679,283, Tonar et al. Pat. No.5,682,267; Tonar et al. Pat. No. 5,689,370; Tonar et al. Pat. No.5,888,431; Bechtel et al. U.S. Pat. Nos. 5,451,822; 5,956,012; PCTApplication Serial No. PCT/US97/16946; and U.S. patent application Ser.No. 09/236,969, now abandoned, the disclosures of which are incorporatedherein by reference thereto.

The forward ambient light sensor 1107 and rear glare sensor 1109 measurethe forward and rear light levels, as indicated in step 1401 (FIG. 14).The forward ambient measurement is used to control both the low beam andhigh beam headlamps 1131, 1132 and the electrochromic mirror 1102. Therear glare measurement is used for the control of the electrochromicmirror 1102 reflectivity. The forward ambient light level measurement isaveraged with prior measurements to compute a forward time averagedambient light level. This average light level is computed as the averageof measurements taken over a 25-30 second interval. Responsive thereto,the microcontroller 1105 computes the control state for electrochromicelement 1105 as a function of the light level measured by sensors 1107and 1109 in step 1202. Where the microcontroller 1115 is a HitachiH8S/2128 microcontroller, the electrochromic element drive state can beset by programming a pulse-width modulated (PWM) duty cyclecorresponding to the desired reflectance level of the electrochromicelement into a pulse-width modulated peripheral to the Hitachi H8S/2128microcontroller. This PWM output is then fed to a series drive circuit.If the headlamps 1131, 1132 are not in auto mode as determined in step1403, which is manually set responsive to the signal 1121, themicrocontroller 1105 returns to step 1401, such that the microcontrollerwill continue to control the variable reflectance of the electrochromicelement 1102. Decision 1403 provides the user with a manual override ifthe high beams are ON. Additionally, the high beam automatic controlwill be skipped in step 1403 (the decision will be NO) if the high beamsare not ON.

If it is determined in step 1403 that the automatic mode is active, themicrocontroller 1105 uses the average ambient light level measured instep 1401 to determine whether the ambient light level is below a lowbeam minimum threshold in step 1404. The threshold may, for example, be1-3 lux, and in one implementation was 2 lux. If the ambient light levelis above the threshold, indicating that high beam headlamps would notprovide significant advantage, high beam control will not be used, andthe microcontroller 1105 returns to step 1401. If the ambient lightlevel is below the low beam minimum, for example, below approximately 2lux, the use of high beam headlamps may be desired. In this case, themicrocontroller 1105 will operate to control the headlamps 1131, 1132.In addition to the average ambient light level discussed above, it isalso advantageous to consider the instantaneous ambient light level.Should the instantaneous ambient light level suddenly drop to a very lowvalue, for example, less than 0.5 lux, automatic high beam operation maybegin immediately rather than waiting for the average ambient lightlevel to reach the threshold for operation of the high beams. Thissituation may occur when a vehicle sitting under a well-lit intersectionsuddenly crosses the intersection into a dark street where high beamoperation is desired immediately. The microcontroller 1105 analyzesimages of the forward scene acquired by image sensor 201 to detect thepresence of oncoming or preceding vehicles as indicated in step 1405.Based upon the results of step 1405, the microcontroller sets thecontrol state of the headlamps in step 1406. Setting the control staterequires setting a duty cycle for the pulse drive in the preferredembodiment. The Hitachi H8S/2128 microcontroller includes timer/counterperipherals which can be used to generate pulse-width-modulated signals1206 and 1207. In some vehicles, the low beam headlamps will always beON regardless of the state of the high beams. In such a vehicle, onlythe duty cycle of the high beams will be varied. Other vehicles willhave the low beams OFF when the high beams are ON, in which case theduty cycle of the low beams will be reduced as the high beam duty cycleis increased. Control of the vehicle headlights using a PWM signal isdisclosed in U.S. Pat. No. 6,049,171, entitled “CONTINUOUSLY VARIABLEHEADLAMP CONTROL,” filed by Joseph S. Stam et al. on Sep. 18, 1998, thedisclosure of which is incorporated herein by reference.

Step 1405, which is the process of acquiring images in the forward fieldof the vehicle and analyzing such images, is described in greater detailwith reference to FIG. 15. A first pair of images are acquired in step1501 through both the red lens 209 and the cyan lens 208, correspondingto the two fields 702 and 703 shown in FIG. 7. The field of view of theresulting images is approximately 25° horizontally and 15° verticallyusing the 64 by 26 pixels, the lens optics, and the far field baffledescribed above. These images are taken at a low sensitivity.Sensitivity of the image sensor 201 may, for example, be dictated by theframe exposure time, the analog amplifier gain, and the DAC high and lowreferences. The image sensor should be just sensitive enough to imageoncoming headlamps at the maximum distance for which the controlledvehicle's headlamps should be dimmed. These images will be sufficient todetect oncoming headlamps at any distance of interest and nearby taillamps without being washed out by bright headlamps or other noise lightsources. In this mode, the sensor should not be sensitive enough todetect reflections off signs or reflectors except in rare cases wherethe reflecting object is very near to the controlled vehicle. Duringdark ambient light conditions, this sensitivity will be low enough todetect only lighted objects.

The sensitivity of the image sensor when acquiring the images in step1501 may also be varied according to whether the high beams arecurrently ON. When controlling high beams manually, a driver willtypically wait until an oncoming vehicle 105 (FIG. 1) or a precedingvehicle 110 is almost close enough for the controlled vehicle's highbeams to be annoying before dimming the headlamps 111. However, if thehigh beams are OFF, most drivers will not activate their high beamheadlamps even if an oncoming vehicle 105 is at a great distance. Thisis in anticipation that the oncoming vehicle 105 will soon come within adistance where the controlled vehicle's high beam headlamps will annoythe oncoming driver, such that the driver of the controlled vehicle 100will have to turn the high beams OFF shortly after they were activated.To partially mimic this behavior, a higher sensitivity image is acquiredif the high beams are OFF, enabling detection of vehicles at a greaterdistance, than if the high beams are ON. For example, the image sensorwhen the high beams are OFF can have 50% greater sensitivity than whenthe high beams are ON.

The images are analyzed in step 1502 (FIG. 15) to locate any lightsources captured in the images. In step 1503, the properties of thelight sources detected are analyzed to determine if they are fromoncoming vehicles, preceding vehicles, or other objects. If a lightsource from a preceding vehicle 110 is bright enough to indicate thatthe high beams should be dim, the control process proceeds to step 1510and the high beam state is set.

If no vehicles are detected in step 1503, a second pair of images aretaken through lens 208, 209 at a greater sensitivity. First, adetermination of the state of the high beams is made in step 1505. Thisdetermination is made because the sensitivity of the second pair ofimages may be five to ten times the sensitivity of the first pair. Witha higher sensitivity image, more nuisance light sources are likely to beimaged by the image sensor 201. These nuisances are typically caused byreflections off road signs or road reflectors, which become much morepronounced with the high beams ON. When the high beams are ON, it isadvantageous to limit the forward field of view to the area directly infront of the controlled vehicle such that it is unlikely that reflectorsor reflective signs will be in the field of view. An ideal narrowedfield of view is about 13° horizontally, which is achieved by areduction of the width of regions 702 and 703 (FIG. 7) to about 35pixels. If the high beams are OFF, an image with the same field of viewas the low sensitivity images acquired in step 1501 can be used sincethe reflections of low beam headlamps off of signs and reflectors aremuch less bright than those when high beams are used. Thus, the decisionstep 1505 is used to select either a narrow field of view image in step1506 or a wide field of view in step 1507. For either field of view, apair of images will be taken. As described above with respect to theacquisition of low sensitivity images in step 1501, the sensitivity ofthe high sensitivity images may also be varied according to the state ofthe high beam headlamps to provide additional control in avoidingnuisance light sources.

As with the low sensitivity images, the high sensitivity images areanalyzed to detect light sources in step 1508. A determination is madeif any of these light sources represent a vehicle close enough to thecontrolled vehicle to require dimming of the high beam headlamps.

Additional images, with greater sensitivity and/or different fields ofview, may be acquired in addition to those mentioned above. Additionalimages may be necessary depending on the dynamic range of the imagesensor. For example, it may be necessary to acquire a very highsensitivity image with a very narrow field of view to detect a precedingcar's tail lamps at a great distance. Alternatively, should the imagesensor have sufficient dynamic range, only one pair of images at onesensitivity may be needed. It is advantageous to use three sets ofimages, a low sensitivity set, a medium sensitivity set, and a highsensitivity set. The medium sensitivity set has about 5 times thesensitivity of the low gain and the high gain has about 20 times thesensitivity of the low gain. The low gain image set is primarilyutilized for detection of headlamps while the medium and high gainimages are primarily utilized for detection of taillamps.

Depending on the quantum efficiency of the image sensor at differentwavelengths of light, and the filter characteristics of the filters usedfor the lens elements 208 and 209, it may be advantageous to use adifferent sensitivity for the two regions 702 and 703. The timing andcontrol circuitry 309 described in the U.S. Pat. No. 5,990,469 can beenhanced to provide the capability to acquire two different windows ofpixels simultaneously and use different analog gains for each window.This is accomplished by adding a register which contains the gain valuesfor the analog amplifier used during acquisition of each subwindow. Inthis way, the relative sensitivity of the two regions can beautomatically balanced to provide a similar output when a white lightsource is imaged through both lens elements. This may be of particularadvantage if the image for region 703 is acquired without filtering thelight rays, for example, by using a clear lens instead of a cyan filterfor rays passing through lens element 208. For this lens set, the pixelsin region 702 will receive approximately 3 times as much light whenimaging a white light source as those pixels in region 703 which receivelight that has passed through a red filter. The analog gain can be set 3times as high for pixels in red filtered region 703 to balance theoutput between the two regions.

The analysis of images to detect light sources indicated in steps 1502and 1508 is described with reference to FIG. 16. Analysis begins withthe image in region 703 acquired through the red lens. It isadvantageous to begin with the red filtered image because severalnuisance light sources do not admit a significant amount of red light.These nuisance light sources include mercury vapor streetlights, greentraffic lights, and reflections off green and blue highway signs.Therefore, a number of potential nuisance light sources are eliminatedfrom consideration. Pixel locations are referred to by X and Ycoordinates with the 0,0 pixel location corresponding to a top leftpixel. Beginning with the 0.0 pixel 1401 and raster scanning through theimage, each pixel is compared to a minimum threshold in step 1602. Theminimum pixel threshold dictates the faintest objects in the image thatmay be of interest. If the current pixel is below the pixel thresholdand it is not the last pixel in the red image window, as determined instep 1603, analysis proceeds to the next pixel as indicated in step1604. The next pixel location is determined by raster scanning throughthe image by first incrementing the X coordinate and examining pixels tothe right until the last pixel in the row is reached, and thenproceeding to the first pixel in the next row.

If it is determined that the current pixel value is greater than theminimum threshold, a seed fill analysis algorithm is entered in whichstep the size, brightness, and other parameters of the identified lightsource are determined as indicated in step 1605. The seed fill algorithmis used to identify the pixels of the image sensor associated with acommon light source, and thus identify associated pixels meeting a pixelcriteria. This can be accomplished by identifying contiguous pixelsexceeding their respective threshold levels. Upon completion of the seedfill algorithm, the properties of the light source are added to thelight source list in step 1606 for later analysis (steps 1503 and 1509in FIG. 15) to determine if certain conditions are met, which conditionsare used to identify whether the light source represents an oncoming orpreceding vehicle. A counter of the number of sources in the list isthen incremented as indicated in step 1607. The microcontroller thengoes to step 1603.

If it is determined in step 1603 that the last pixel in the red imagehas been examined, the microcontroller 1105 determines whether any lightsources were detected, as indicated in step 1608. If no light source isdetected, the analysis of the image terminates as indicated at step1609. If one or more light sources are detected through the red lens209, the cyan or clear image window 702 is analyzed to determine thebrightness of those light sources as imaged through the other lens 208,in order to determine the relative color of the light sources. In thissituation, the “brightness” of the sources refers to the sum of the greyscale values of all the pixels imaging the source; a value computed bythe seed fill algorithm. The first source on the list is analyzed instep 1610. A seed fill algorithm is executed in step 1611, with the cyanimage starting with the pixel having the same coordinates (relative tothe upper left of the window) as the center of the light source detectedin the red image. In this manner, only those pixels identified with alight source viewed through the lens associated with the red filter willbe analyzed as viewed through the other filter, which is advantageous,as many nuisance light sources which would otherwise be analyzed whenviewed through a clear or cyan filter will be removed by the red filter.By only considering light sources identified through the red filter, theamount of memory required to store information associated with lightsources viewed through the other filter will be reduced. The ratio ofthe brightness of the source in the red image 703 to the brightness ofthe source in the other image 702 is stored in the light list, asindicated in step 1612, along with the other parameters computed in step1605 for the current light source. This procedure continues for othersources in the light list 1613 until the red to cyan ratios are computedfor all sources in the list 1615, at which point the analysis terminatesat step 1614.

It will be recognized that where it is desirable to count the number oflight sources to determine the type of driving environment, and inparticular to identify city streets or country roads, it may bedesirable to count all of the light sources viewed through the cyan orclear filter. In this way, nuisance light sources can be counted.Alternatively, the number of light sources viewed through the red filtercan be counted for purposes of inhibiting turning ON the high beams if athreshold number of sources are identified.

The seed fill algorithm used in steps 1605 and 1611 is shown in FIG. 17a. The outer section of the seed fill algorithm is entered with thecurrent pixel value at step 1605. The outer section of the seed fillalgorithm is executed only once at each step 1605, while the innerrecursive seed fill algorithm is entered many times until all contiguouspixels meeting a pixel criteria are identified. After the entry step1701 of the outer section of the seed fill algorithm, several variablesare initiated as indicated in step 1702. The variables XAVG and YAVG areset to zero. These variables will be used to compute the average X and Ycoordinates of the pixels imaging a light source, which averagecoordinates will together correspond to the center of the light source.The TOTALGV variable is used to sum the grey scale values of all thepixels imaging the source. This value will define the brightness of thesource. The SIZE variable is used to tally the total number of pixelsimaging the source. The MAX variable stores the maximum grey scale valueof any pixel imaging the source. A CALLS variable is used to limit thenumber of recursive calls to the recursive inner seed fill function toprevent memory overflow as well as for tracking.

The inner seed fill algorithm is first entered in step 1703. The startof the inner recursive seed fill function for the first and subsequentcalls to the seed fill function is indicated by block 1704. The firststep in the inner seed fill function is to increment the CALLS variableas indicated in step 1705. Next, microcontroller 1105 determines if theCALLS variable is greater than the maximum allowable number of recursivecalls as determined in step 1706. The number of recursive calls islimited by the amount of memory available for storing light sourceinformation. If it is determined that the number of recursive callsexceeds the threshold, the recursive function proceeds to step 1719,wherein the counter is decremented and then returns to the step fromwhich the recursive function was called. Should this occur, a flag isset indicating that the current light source had too many recursivecalls, and parameters computed by the seed fill algorithm will beincorrect. This prevents too many levels of recursion from occurringwhich would overflow the memory of the microcontroller.

If the decision in step 1706 is that the CALL variable is not greaterthan the maximum allowable, the microcontroller 1105 next compares thecurrent pixel with a minimum grey scale threshold in step 1707 todetermine if the pixel is significantly bright to be included in thelight source. The threshold considered in step 1707 can be constant orvary by position. If it varies, the variable thresholds may be stored ina spatial look-up table having respective pixel thresholds stored foreach pixel, or each region of the image sensor. Where the threshold isvariable by position, it is envisioned that where a pixel at onelocation exceeds its associated pixel threshold, the controller cancontinue to use that threshold for adjacent pixels while searching for acontiguous group of pixels associated with a single light source, or thecontroller can continue to use the respective pixel thresholds stored inthe spatial look-up table.

If the condition in step 1707 is not met, this inner recursive seed fillfunction terminates by advancing to step 1719. If the pixel has a highenough grey scale value as determined in step 1707, its grey scale valueis added to the global TOTALGV variable 1708, its X and Y coordinatesare added to the XAVG and YAVG variables 1709 and 1710, and the sizevariable is incremented in step 1711. If the grey scale value of thepixel is greater than any other pixel encountered in the current seedfill, as determined in step 1712, the MAX value is set to this greyscale value 1713.

Following a no decision in step 1712, the grey scale value of the pixelis set to 0 to prevent further recursive calls from including thispixel. If a future recursive call occurs at this pixel, and the pixel isnot zeroed, the pixel will be added the second time through. By zeroingthe pixel, it will not be added again as the pixel's grey scale value isno longer greater than the minimum threshold. Additionally, this pixelwill be ignored in step 1602 should it be encountered again whilescanning the image during analysis.

The inner recursive seed fill algorithm next proceeds to recursivelycall to itself until it has looked to the right, left, above, and beloweach pixel until all of the contiguous pixels exceeding the minimumpixel threshold value in decision step 1707 are considered. Step 1715represents returning to step 1704 for the pixel to the right. Themicrocontroller will continue to look at pixels to the right until itreaches a pixel that does not meet the criteria of decision step 1706 or1707. Step 1716 represents returning to step 1704 for the pixel to theleft. Step 1717 represents returning to step 1704 to look at the pixelabove. Step 1718 represents returning to step 1704 to look at the pixelbelow. The microcontroller will then look at the pixel to the left ofthe last pixel that did meet decision step 1707. The processor will lookat pixels adjacent each pixel exceeding the threshold of step 1707should the neighboring pixels exist (i.e., the immediate pixel, is notan edge pixel). Step 1719 decrements the CALLS variable each time thestep is executed, and the microcontroller will return to the callingprogram until the CALLS value reaches 0. Returning to the function thatcalled it may be another instance of the inner recursive function, orshould this be the initial pixel examined, to the outer recursivealgorithm 1721.

An example of how the inner and outer seed fill algorithms operate willnow be described with reference to FIG. 17 b. The example is made withrespect to an exemplary very small image sensor having 30 pixels. Pixels4, 9, 10, 11, 14, 15, 16, 17, 18, 21, 22, 23, and 28 exceed thethreshold in step 1707 in this example. Additionally, the number ofcalls required does not exceed the threshold in step 1706. The imagearray 301 is impacted by a light source 1751, indicated by contour 1751.The microcontroller will operate as follows in evaluating the pixels.For pixel 1, the microcontroller 1105 will enter the seed fill algorithmat step 1701, initialize the variables in step 1702, and set the currentpixel in step 1703. The microcontroller will next enter the inner seedfill function in step 1704. The CALLS variable will be incremented to 1in step 1705, which is below the Maximum Calls threshold. Because thereis no light on the pixel, the minimum threshold is not exceeded and themicrocontroller will go to step 1719, decrement the CALLS variable instep 1720, and because this is the first time through the inner seedfill program, the microcontroller will continue to steps 1721-1723. Theprocess will be repeated for pixels 2 and 3, which are both below theminimum pixel threshold used in step 1707.

When the microcontroller gets to pixel 4, it will enter the outer seedfill at step 1701, set the variables to zero in step 1702, set thecurrent pixel to pixel 4 in step 1703, and enter the inner seed fillalgorithm. The CALLS variable will be incremented to 1, as it is thefirst pixel in this outer seed fill. The CALLS variable will be lessthan the Maximum Calls threshold and the pixel's grey scale value willexceed the minimum threshold. Accordingly, the grey value will be addedto TOTALGV, the pixel coordinates will be added to those that will beconsidered for XAVG and YAVG, and the SIZE will be incremented such thatthe size will be 1. The grey value for pixel 4 will be the MAX, as it isthe only grey value in this outer seed fill. The grey value will be setto zero for pixel 4 in step 1714.

The microcontroller will then identify all of the contiguous pixels thatexceed the threshold set in step 1707. In particular, through the innerseed fill routine disclosed in FIG. 17 a, the microcontroller will addthe pixels as follows. Pixel 4 is added first as it is the first with agrey scale value greater than the threshold (“the threshold” in thisparagraph referring to the minimum threshold in step 1707). The programthen calls the recursive function for pixel 5 to the right, which is notadded as it is below the threshold (as used in this paragraph, “added”means the pixel's coordinates are added to XAVG and YAVG, the pixel'sgrey scale value is added to TOTALGV, SIZE is incremented, and thepixel's grey scale value becomes the maximum pixel grey scale value ifit exceeds MAX). Pixel 3 to the left is called next, and it is not addedfor the same reason. There is no pixel above pixel 4. Accordingly, therecursive function is next called for pixel 10 in step 1718. Pixel 10will be added as its value is greater than the threshold. Themicrocontroller 1105 will then look to the right of pixel 10, namely atpixel 11, which is greater than the threshold, so it will be added. Themicrocontroller will then look to the right of pixel 11, which is pixel12. Pixel 12 will not be added as its grey scale value is below thethreshold. Looking to the left of pixel 11, pixel 10 will not be addedas it was zeroed when it was added previously. Looking above pixel 11,pixel 5 will not be added. Looking below pixel 11, pixel 17 will beadded. Next, the recursive routine will be called for the pixel to theright of pixel 17. Pixel 18 will be added. Moving on to the recursiveroutine for pixel 18, there is no pixel to the right of pixel 18.Looking to the left of pixel 18, pixel 17 will not be added as it waszeroed when it was added. Looking above pixel 18, pixel 12 will not beadded. Looking below pixel 18, pixel 24 will not be added as it is belowthe threshold. Moving back to the previous recursive function that wasnot exhausted, the microcontroller 1105 will move back and look to theleft of pixel 17, which is pixel 16. Pixel 16 is added. Calling therecursive function again, the microcontroller will look to the right ofpixel 16. Pixel 17 will not be added as it was cleared after it waspreviously added. Looking to the left of pixel 16, pixel 15 is added.Calling the recursive function from pixel 15, the microcontroller willlook to the right at pixel 16. Pixel 16 is not added, as it was zeroedafter it was previously added. Looking to the left, pixel 14 is added.Calling the recursive function from pixel 14, the microcontroller willlook to the right at pixel 15, which will not be added. Looking to theleft of pixel 14, pixel 13 is not added. Looking above pixel 14, pixel 8is not added. Looking below pixel 14, pixel 20 is not added. Moving backto the function that called the recursive function for pixel 14, themicrocontroller will look above pixel 15 at pixel 9. Pixel 9 is added.Calling the recursive function for pixel 9, looking to the right, pixel10 is not added. Moving to the left, pixel 8 is not added as it is belowthe threshold. Looking above, pixel 3 is not added. Looking down, pixel15 is not added as it was previously cleared. Moving back to thefunction that called the recursive function for pixel 9 has themicrocontroller looking at the pixel below pixel 15. Pixel 21 is thenadded. Starting the recursive function for pixel 21, looking to theright, pixel 22 is added. Starting the recursive function for pixel 22,pixel 23 is added as it exceeds the threshold. Starting the recursivefunction for pixel 23, pixel 24 is not added as it is below threshold.Moving to the left, pixel 22 is not added as it was zeroed. Lookingabove, pixel 17 is not added as it was zeroed after it was added.Looking down, pixel 29 is not added as it is below the threshold. Movingback to the function that called the recursive function for pixel 23,microcontroller 1105 now looks to the left of pixel 22. Pixel 21 is notadded as it was cleared. Looking above pixel 22, pixel 16 is not added.Looking below pixel 22, pixel 28 is added. Performing the recursivefunction for pixel 28, pixel 29 to the right is not added, pixel 27 tothe left is not added, pixel 22 above is not added, and there is nopixel below pixel 28. Moving back one function, the recursive functionfor pixel 22 was completed, so the microcontroller returns to pixel 21.Looking to the left of pixel 21, pixel 20 is not added. Pixel 15 abovepixel 21 is not added and pixel 27 below pixel 21 is not added. Therecursive function for pixel 15 was exhausted, so the microcontrollerlooks above pixel 16 to pixel 10, which is not added. Looking belowpixel 16, pixel 22 is not added. Moving back one function has themicrocontroller looking above pixel 17 at pixel 11, which is not added.Looking below pixel 17, pixel 23 is not added. If the recursive functionfor pixel 11 was completed, the microcontroller returns to the lastnon-exhausted pixel of the contiguous lighted pixels. Themicrocontroller looks to the left of pixel 10, and pixel 9 is not added.The microcontroller looks above pixel 10, and pixel 4 is not added.Looking below, pixel 16 is not added. The inner seed fill is complete.

After the last pixel is added, YAVG, XAVG and size are used to selectthe center of the light in steps 1722 and 1723.

The seed fill scheme just described is the preferred seed fill algorithmfor analysis of the red image 703. It requires some modification for theother image 702. This need can be seen from the following example. Wherea pair of oncoming headlights is identified as two separate lights inthe red image, it is possible that these sources may contain more lightin the cyan half of the spectrum than the red half (as is the case withHigh Intensity Discharge headlamps). In this case, what was detected astwo separate light sources in the red image can bloom into one source inthe cyan image. Thus, there would only be one source in the cyan imageto correspond to the two light sources in the red image. After step 1611is completed for the first source in the red image, the bloomed cyanimage pixels would have been cleared in step 1714, such that they wouldnot be available for the analysis of the second light source in the redimage. Thus, after the source in the cyan image is determined tocorrespond to the first source in the red image, there would be nosource on the cyan image to correspond to second source in the redimage. To prevent this, it is useful to preserve the image memory forthe cyan image rather than setting the grey scale value to zero as instep 1714 so that the bloomed source is detected as corresponding toboth of the sources in the red image.

Since the red image has already been processed, the memory that storedthe red image can be used as a map to indicate which pixels in the cyanimage have already been processed. All pixels in the red image memorywill have been set to grey scale values less than the minimum thresholdas a result of the processing. The pixels that were identified as lightsources where zeroed whereas the light sources that were not clearedwill have some low grey scale value. This characteristic can be used toset the pixels in the red image memory to serve as markers forprocessing pixels in the cyan image. In particular, when the seed fillalgorithm is executed for the cyan image, step 1714 is replaced by astep that sets the value of the pixel in the red image memorycorresponding to the currently analyzed cyan pixel to a marker value.This marker value could be, for example, 255 minus the index of thecurrent light source in the light list. The number 255 is the largestvalue that can be stored for a grey scale value. In this way, there willbe a unique marker stored in the red image memory for pixels analyzedeach time step 1611 is executed for a light on the list. In addition tothe above change to step 1714, a test must be added after step 1707 todetermine if the value of the pixel stored in the red image memorycorresponding to the currently analyzed pixel is equal to the markervalue for the current light source index, indicating that this pixel hasalready been recursively visited, in which case the microcontrollerwould go to step 1719.

Steps 1501 and 1502 for the red image in region 703 and correspondingsteps 1506, 1507 and 1508 for the other image in region 702, have nowbeen explained in detail. In steps 1503 and 1509, the light source listis analyzed to determine if any of the identified light sources indicatethe presence of an oncoming or preceding vehicle. FIG. 18 is a flowdiagram of the series of tests applied to the information gathered oneach light source in the light list during the previous analysis stepsto determine the type of light source. The tests in FIG. 18 are appliedto each source in the list independently.

First, a check is made to see if the recursive seed fill analysis ofthis light source exceeded the maximum number of subsequent recursivecalls allowed as determined in step 1801. If so, the light source islabeled an extremely bright light in step 1803 since the number ofpixels in the source must be large to cause a large number of subsequentrecursive calls. Next, the TOTALGV variable is compared to avery-bright-light threshold 1802. If TOTALGV exceeds this threshold, thelight is also labeled Extremely Bright in step 1803.

If neither of the conditions 1801 or 1802 are met, the light source isanalyzed to determine if it has a 60 Hz alternating current (AC)intensity component, indicating that the light source is powered by anAC source, in step 1804, to distinguish it from vehicles which arepowered by DC sources. Many streetlights, such as high-pressure sodiumand mercury vapor lights, can be effectively distinguished from vehicleheadlamps in this way. To detect the AC component, a series of eight 3×3pixel images are acquired at 480 frames per second, for example. The 3×3pixel window is centered on the center of the light source beinginvestigated. The sums of the nine pixels in each of the eight framesare stored as eight values. These eight values represent the brightnessof the source at ¼ cycle intervals over two cycles. The magnitude of the60 Hz Fourier series component of these eight samples can be computed bythe following formula:AC=(A ² +B ²)^(1/2)A=1.0*(F1−FMIN)+(−1.0)*(F3−FMIN)+1.0*(F5−FMIN)+(−1.0)*(F7−FMIN)B=1.0*(F0−FMIN)+(−1.0)*(F2−FMIN)+1.0*(F4−FMIN)+(−1.0)*(F6−FMIN)where F1 to F7 refer to the eight summed grey scale values of each ofeight frames and FMIN refers to the minimum value of F1 through F7. Thevalue AC is divided by the mean grey value of F1 to F8 to give the ACcomponent of interest in step 1804. The scheme is of particularconvenience since the sampling rate is exactly four times the frequencyof modulation analyzed. If this were not the case, the coefficients foreach of F1 through F7 would have to be changed to accommodate this.Additionally, if the system were to be used in countries where the ACpower is not at a 60 Hz frequency, the sampling rate or coefficientswould be adjusted.

While the AC detection scheme is described for only one window, it isadvantageous to perform the above analysis by imaging the source throughboth the cyan and red filters, or the red and clear filters, and thenusing the maximum computed AC component from the two sample sets todetermine the AC component for step 1804. This allows for accuratemeasurement of sources that have either strong short or long wavelengthcomponents in their spectrum. If the light source AC component isgreater than the minimum threshold as determined in step 1805, the lightsource is determined to be an AC source as indicated in step 1806.

The next step is to distinguish light sources which are reflections offof road signs or other large objects. Road signs are large compared tothe size of the headlamp or tail lamp. As a result, they tend to beimaged onto many more pixels than a vehicle lamp, unless the lamp is sobright as to bloom in the image. Sign reflection sources may have alarge number of pixels, indicated by the SIZE variable, but a loweraverage grey scale value than vehicle lamps which have a much higherintensity. If the SIZE variable exceeds a given size threshold asdetermined in step 1807, the average grey scale value (TOTALGV/SIZE) iscompared against an average pixel level threshold, which is the maximumsign average grey scale value, in step 1808. If the average grey scalevalue is less than this threshold, the light source is determined to bea reflection off of a sign as indicated in step 1809.

An additional check for yellow signs, which typically occurs in thefront of the vehicle when turning through a curved road, is to measurethe color of bright objects and determine if they are likely yellow.Even when using the low gain image, the reflection off of a nearby signmay saturate several pixels in both subwindows and make relative colordiscrimination difficult. However, the same high-speed images acquiredwhen performing an AC validation can be used to accurately determine therelative color even for bright images. This is accomplished by lookingat the average value of the F1 through F7 frames mentioned hereinabovethrough each lens. The ratio of the average of the F1 to F7 framesviewed through the red lens to the average of the F1 through F7 framesviewed through the cyan or clear lens is computed. This ratio will behigher for yellow signs than for white lights, but not as high as fortail lamps. If this ratio is within a certain range indicating that itis likely a yellow sign, this object can be ignored or the threshold fordimming for this object can be decreased, allowing only extremely brightyellow objects to dim the high beams. By allowing the system to respondto bright yellow objects by dimming the high beams, the headlight dimmerwill respond to a light source in the event that a headlamp ismisdiagnosed as a yellow sign.

Once potential nuisance sources are filtered out, the light source canbe identified as a headlamp or a tail lamp. The red to cyan ratiocomputed in step 1612 is used to determine if the source has sufficientred component to be classified as a tail lamp in step 1810. If thisratio is lower than a tail lamp redness threshold, control proceeds tostep 1811, where the threshold for consideration as a headlamp isdetermined. This threshold can be determined in a number of ways, butthe most convenient method is to use a two-dimensional look-up table ifthe microcontroller has sufficient read-only memory (ROM) to accommodatethe table, as is the case with the Hitachi H8S/2128 exemplified herein.

The center of the light source is used as the index into the look-uptable to determine the threshold. The look-up tables are created toprovide different threshold for different regions of the field of view.For example, the thresholds for regions of the field of view directly infront of the vehicle are lower than those used for regions off to theside. The look-up table can be optimized for the particular vehicle'shigh beam emission pattern to determine how annoying the high beams willbe to oncoming and preceding traffic at various angles. Ideally, thelook-up table used when the vehicle's high beams are activated isdifferent from when they are not. When high beams are off, thethresholds to the side of the vehicle can be lower than they are whenhigh beams are activated. This will prevent the high beams from comingON when there is a vehicle in front of, but off at an angle to, thecontrolled vehicle, such as happens when on a curve on an expressway.Also, different lookup tables are used for high and low sensitivityimages for much of the same reasons. In fact, the lookup tables mayindicate that, for certain regions, or all regions of the field of viewin high sensitivity image, that head lamps be ignored completely. A highsensitivity image is only analyzed if the low sensitivity did not detectoncoming or preceding vehicles, so non-red light sources in a highsensitivity image are most likely very distant headlamps or nuisancesources. Therefore, the high sensitivity images may only be analyzed forred sources.

In step 1812, the TOTALGV variable, which is the total pixel level, iscompared to the threshold total pixel level for headlights as determinedfrom the appropriate look-up table. If this variable is greater than thethreshold, the value is compared to the threshold multiplied by a brightmultiplier in step 1813. If TOTALGV is greater than this value, thelight source is determined to be a bright headlamp in step 1815. If not,the light source is a headlamp as indicated in step 1814. The need todistinguish between headlamps and bright headlamps will become clearerin the discussion of the control state of the headlamps. As analternative to comparing TOTALGV to a threshold times a brightmultiplier, yet another look-up table can be provided to dictatethresholds for bright headlamps.

A procedure similar to that just described is performed in step 1816 ifthe light source is determined to be red. A series of lookup tables areprovided for taillight thresholds and the appropriate lookup table isused to determine the threshold for the given light source location. In1817, the total pixel level for taillight threshold is compared to theTOTALGV variable to determine to light sources bright enough to be ofinterest. If so, the light source is labeled as a tail light in step1819. If not, the light source is ignored in step 1820.

Should the cyan lens element 301 be replaced by a clear lens, adifferent ratio threshold is used in step 1810. In this case, a redlight will have approximately the same brightness in both images, but awhite or other color light will be substantially brighter when imagedthrough the clear lens. This ratio is determined based on the analoggain settings for both the red and cyan (or clear) images.

Once the images of the forward field have been acquired and analyzed instep 1405, the state of the headlamps must be determined in step 1406.This is best described by considering a state machine implemented in themicrocontroller 1105, which state machine is illustrated in FIG. 19. Inthe following example, it is assumed that headlamp drive 902 is a pulsewidth modulator drive and the high beam brightness can be varied bychanging the duty cycle of the pulse width modulator drive. It is alsoassumed that the low beam headlamps remain on at 100% duty cycle at alltimes. Each of the states in FIG. 19 represents a control state of thehigh beam headlamps. The duty cycle of the high beam headlamps is set tothe value indicated by the current control state. The distribution ofthe duty cycles amongst the control states is non-linear to compensatefor the fact that headlamp brightness is a non-linear function of dutycycle and to provide the appearance of a constant percent change fromcycle to cycle. Also, there are several states for both 0% and 100% dutycycle, indicating that several states must be traversed before theheadlamps begin to go ON or OFF, providing a time delay verificationinsuring that the detected light source is persistent over severalimages. The number of states is exemplary only, and those skilled in theart will recognize that it may vary depending on the desired fade ON andOFF rate of the high beams and time between cycles. Additionally, thebright light indicator (which is typically located on the vehicle'sdashboard) fades ON as the states move from state 11 to state 14. Thebright light indicator fades OFF as the states move from state 7 tostate 4. This provides some hysteresis to avoid flashing the indicatorON and OFF. Alternatively, if the indicator does not fade ON and OFF,the state at which the bright light indicator turns ON will preferablybe a higher state than the state at which the bright light indicatorturns OFF. In either case, hysteresis is provided for the headlampcontrol.

After each control cycle, the headlamp control state can remain the sameor move to a different state. The rules for moving between states aredescribed with reference to FIG. 20. First, if any of the lights in thelight list were determined to have been “Extremely Bright” lights instep 2001, the current control state is immediately set to a 0 state instep 2002. This behavior allows bypass of the fade out feature and rapidresponse to the sudden appearance of headlamps in the forward field ofview, such as that which happens when driving over a hill. Step 2003determines whether a headlamp or tail lamp was detected in the lightsource list. If not, control proceeds to step 2004, wherein it isdetermined whether the state was decremented in the last control cycle.

One of the advantages of the variable beam configuration is the abilityto fade in and out high beams. If a reflection from a sign or reflectoris misdiagnosed as a vehicle lamp, the high beams will begin to dim.When they do, the reflected source will no longer be present in theimage and the high beams will fade back on, likely without bothering thedriver. To prevent rapid oscillation between increasing and decreasingheadlamp beam brightness, the control state must rest in the currentstate for one cycle before changing direction. Therefore, step 2004considers the action during the previous cycle and the system willremain in the current state as indicated in step 2005 if the previousaction was to decrement states.

If the action during the previous cycle was not to decrement states,vehicle speed is considered next in step 2006. If a vehicle is stoppedat an intersection, it will annoy the driver if the high beams come ON,dim with a passing car, and then come ON again. To prevent this, thehigh beams are prevented from coming ON if a vehicle is traveling belowa certain speed, for example 20-mph. A decrease in speed below theminimum speed threshold will not necessarily cause the high beams to dimunless there is an oncoming or preceding car present. Alternatively, thehigh beams can be configured to fade OFF every time the vehicle comes toa stop and then fade ON again once the vehicle begins moving, providedno other vehicles are present. In step 2007, the number of lights in thelight list (counted in step 1607) is checked as an indication of whetherthe driver is driving in city conditions. If there are a large number oflights in the forward field of view, the high beams will not be turnedON. The number of the lights that are alternating current sources mayalso be considered.

If the criteria of steps 2004, 2006, and 2007 are met, the currentcontrol state is incremented one state in step 2008. If this actionresults in the current control state exceeding state 12 as determined instep 2009, the current control state is set to 15 in step 2010 forcingseveral time verification states before the beams can be dimmed again.

Should the microcontroller 1105 detect a headlamp or tail lamp in step2003, the microcontroller determines whether a bright headlamp wasdetected in step 2011. If not, the prior cycle action is considered instep 2012 to prevent rapid oscillation between incrementing anddecrementing to avoid rapid ongoing oscillations between states. If theprior action was not to increment, the current control state isdecremented in step 2014. If the new control state is below state three,as determined in step 2015, the current control state is set to 0 instep 2016.

Finally, if a bright headlamp is detected in step 2011, it is desirableto more rapidly fade out the high beams by decrementing 2 states in step2018, provided the previous action was not to increment in step 2017, inwhich case the current state is decremented only one state.

A large number of variations of the scheme just described are possible.For example, more states can be added to increase or decrease the timeit takes to fade the high beams in and out. The number of requiredstates will depend upon the image acquisition and analysis cycle time.Ideally, the fade in/out period is about one to two seconds. Anotheralternative is to decrement states as a function of the brightest lightsource detected in the light list rather than to decrement a singlestate for every cycle. In this way, the brightness of the high beam isadjusted as a function of the distance of an oncoming or precedingvehicle rather than just fading in and out. This is particularlyadvantageous where the control mechanism is to vary the beam angle ofthe high beams rather than the intensity of the beams. Yet anotheralternative is to decrement states as a function of the current speed ofthe controlled vehicle. The rate at which states are decremented couldincrease at high vehicle speeds, since oncoming cars will overtake thecontrolled vehicle at a more rapid rate. Yet another alternative is todecrement states as a vehicle slows down. This would allow the highbeams to fade out as a vehicle comes to a stop, a feature that may bedesirable for some drivers. Finally, it should be noted that the use ofdiscrete states is only exemplary. The intensity and/or aim of the highbeam headlamps can be controlled by continuum of values from fully ON tofully OFF.

The previous discussions described in detail the operation of one cycleof the headlamp dimmer control sequence. This sequence is repeatedindefinitely as long as the device is on. Depending on the time tocomplete one cycle, the above procedure may be interrupted to acquirerear glare sensor measurements for the electrochromic mirror.Additionally, the driver may interrupt the above sequence by activatingthe manual high beam switch generating input signal 1123 (FIG. 11). Thisfeature is necessary to allow the driver to override improper behaviorof the control system or to use the high beams to alert other driversthrough flashing the high beams.

Some additional features may be provided with the above-describedhardware and software configuration. One such feature is daytime runninglights (DRLs). On some vehicles, DRLs are provided by operating the highbeams at a reduced intensity. By using the PWM drive circuitry provided,the high beams can be set to a reduced intensity during daylightconditions. The ambient light sensor 1107 can be used to determinedaylight conditions and switch the headlamps to normal low beamoperation at dusk. In particular, the ambient light level can have oneor more light level thresholds associated therewith. When the ambientlight level is a daytime one threshold, the daytime running lights willbe ON. Below that threshold, but above another lower threshold, the lowbeams can be ON. Below that lower bright activate ambient light levelthreshold, the high beams may be operated automatically if the driverdoes not manually disable high beam operation.

Even without the use of DRLs, it is desirable to have automaticactivation of the low beam headlamps at dusk. This control can beprovided by the use of the ambient light sensor 1107. For betterperformance, an additional light sensor can be provided which senseslight from a direction upwards rather than looking straightforward asthe ambient light sensor 1107 does. Such a sky sensor arrangement isdisclosed in U.S. patent application Ser. No. 09/491,192, entitled“VEHICLE EQUIPMENT CONTROL WITH SEMICONDUCTOR LIGHT SENSORS,” filed byJon H. Bechtel et al. on Jan. 25, 2000, now U.S. Pat. No. 6,379,013, thedisclosure of which is incorporated herein by reference thereto.Alternatively, a few rows of pixels in the image sensor can be used toimage a region of the sky above the horizon. An imaging systemdisclosing such an arrangement is described in U.S. Pat. No. 6,130,421,the disclosure of which is incorporated herein above by reference. Theability to image through both the red and cyan filters can help todistinguish between clear and overcast conditions and to detect asunset.

It is envisioned that the system can detect a tunnel using the imagesensor and a sky sensor. In particular, when a large dark area, which isa contiguous area of dark pixels meeting a size threshold level that islocated in the center of the image, under day ambient light conditions,a potential tunnel condition is detected. If the dark area grows whilethe ambient light conditions continue to sense day ambient lightconditions, the potential tunnel condition will continue. If the imagesensor continues to see a large dark area forward of the vehicle whenthe daylight ambient conditions are no longer detected, the vehicle willbe determined to be in a tunnel, and the headlights will be ON. Theheadlights will remain on until the daylight ambient conditions aredetected, at which time the headlights will be turned OFF and daylightrunning lights will be turned ON if the controlled vehicle has daytimerunning lights.

Speed Varying Threshold

The speed input 1117 can be used in additional ways by microcontroller1115. For example, when driving on an expressway, it is necessary todetect oncoming cars in the opposite lane, which is usually separated bya median. Therefore, the microcontroller must be more sensitive tooncoming headlamps at larger angles than would be necessary forback-road driving where there is no median separating oncoming traffic.Incorporating a wide field of view for back-road driving has thedisadvantage of increasing the likelihood that the image sensor willdetect house lights, which are typically incandescent. Incandescentlight sources will not be filtered by the AC rejection algorithm. Suchhouse lights are not typically present on the side of the freeway. Byincreasing the sensitivity of the device to objects at higher angleswhen the vehicle is traveling at higher speeds, such as speeds in excessof 50 to 60 mph, the system will be able to sense cross-median trafficon expressways without increased sensitivity to house lights on backroads. It is envisioned that the field of view could be increased whenthe vehicle speed exceeds 65 mph and decreased when the vehicle speedpasses below 60 mph.

In particular, it is envisioned that prior to step 1601, additionalsteps can be provided to make the field of view speed sensitive. Thesesteps are illustrated in FIG. 21. The vehicle speed is input in step2101. The microcontroller determines in step 2103 whether the status iscurrently a wide view for expressway driving. If it is not, themicrocontroller 1105 determines whether the speed is greater than 65mph. If not, the status does not change. If the vehicle speed isdetermined to be greater than 65 mph in step 2105, the field of view ofthe image sensor 201 is increased as indicated in step 2107. The fieldof view in the horizontal direction can be increased by 30 to 150percent, and advantageously 60 to 125 percent, relative to the narrowerfield of view at lower speeds. If it was determined in step 2101 thatthe controlled vehicle 100 is in the expressway state, themicrocontroller 1105 determines in step 2109 whether the vehicle speedhas dropped below 45 mph. If it has, the microcontroller reduces thefield of view of image sensor 201 to its narrower non-expressway fieldof view. In one advantageous embodiment, the field of view remains thesame at all speeds, but the thresholds for responding to lights athigher angles are decreased as speed increases. This is advantageousbecause it allows detection of very bright objects at high angles whenthe vehicle is traveling at slow speeds, but it is not sensitive to lessbright objects, such as house lights or street signs, when traveling atslow speeds. In particular, at high speeds, reducing the thresholdsincreases the sensitivity to cross-median traffic. The thresholds arereduced more significantly to the left than to the right sincecross-median traffic is to the left. This property can be reversed forleft-hand drive countries.

It is envisioned that the field of view will be varied by increasing anddecreasing the width of the pixels used by the image sensor in themanner described above with respect to varying the field of view whenthe high beams are ON or OFF.

According to another aspect of the invention, the integration time forthe pixels of the image array sensor can be increased at higher speeds.This increase in integration period increases the sensitivity of theimage array sensor. The system is thus able to detect an oncomingvehicle sooner. This may be particularly advantageous as oncoming carsare likely to be traveling faster when the controlled vehicle istraveling faster. Accordingly, providing the light control system with ahigher sensitivity to shorten the response time will better emulate adesired dimming characteristic of the headlamps by dimming the headlampsat a desired vehicle distance. To provide this function, in step 2107,the microcontroller 1105 increases the sensitivity and/or widens thehorizontal viewing angle, whereas in step 2111, the microcontrollerdecreases the sensitivity and/or narrows the horizontal viewing angle.

It is further envisioned that adjustments can be made to the thresholdin step 1707 as a function of speed. Such a system can be implementedusing a look-up table stored in read only memory. FIG. 22 illustratesregions of the image sensor array 301, with the regions radiatingoutwardly from the center. The regions may not be symmetrical sinceoncoming traffic usually occurs more to the left and signs and nuisancelight sources are more likely to the right in right side drivecountries. The look-up table sets a respective threshold for each of theregions 1-6 with the regions having sequentially, incrementally greaterthresholds moving outwardly from the center. Thus, the center region 1will have the lowest threshold and the outer peripheral 6 will have thehighest threshold. In addition, the thresholds at different angles willvary as a function of speed, such that at higher speeds, the thresholdswill be lower in regions 3 through 5 than they will be at higher speeds.For example, where two fields of view are provided, one table willcontain respective thresholds for pixels in the regions for lower speedsand another table will contain respective thresholds in regions for thehigher speeds. Each table will assign an integration time and/orthresholds for use in analyzing different portions of the field of view.Because the greater the distance from the center, the greater theviewing angle of the scene image, lowering the threshold in the outerregions increases the system's responsiveness to light sources at widerangles to the controlled vehicle.

Either the thresholds or the integration times for the pixel sensors ofthe image array, or both the thresholds and the integration times, canbe changed to increase the sensitivity of the sensor at differentspeeds. Thus, it is envisioned that the field of view can be the same atboth high and low speeds. However, the thresholds or the sensitivity(i.e., the integration period or the amplifier gain) can be altered suchthat at low speeds, images viewed at wide angles will have little impacton the dimming decision whereas at high speeds, the images viewed atwide angles will have substantially more impact on the dimming decisionat high speeds. It is further envisioned that instead of using look-uptables, an algorithm can be used to alter the sensitivity of the lightsensor as a function of the angle of the image being viewed. Such analgorithm can be used to reduce the memory requirements for the systemat the expense of requiring greater processing power to implement thefunction.

Turning

As indicated above, it is disadvantageous to have a wide field of viewwhen traveling slowly, such as occurs when traveling on back roads.However, when turning, it is necessary to have a wide field of view sothat the image sensor can detect other vehicles that will come in frontof the controlled vehicle prior to the controlled vehicle's headlampsstriking the other vehicle. A subroutine that the microcontroller 1105uses for changing the viewing angle of the image sensor is disclosed inFIG. 23. It is envisioned that the subroutine will be executed each timea turn condition is initiated. For example, the microcontroller maydetect any one or more of the following: activation of a vehicle turnsignal alone or in combination with braking; a change in vehicle headingdetected by the compass; global positioning information; and change indirection of the vehicle steering wheel or front tires. Responsive tosuch an event, the microcontroller 1105 will input the criteria to beused in making a decision as to whether the field of view needs to bealtered. For example, a compass input, an input from a turn signal,global positioning system (GPS) receiver information, wheel turnindication from the vehicle steering system, vehicle speed, or acombination of these inputs may be input in step 2301.

In step 2303, the microcontroller 1105 will determine whether thecriteria input in step 2301 indicates that the field of view should bealtered. For example, when the vehicle turn signal is ON, and thevehicle is slowing down, the viewing angle sensitivity can be altered inthe direction of the turn signal. Alternatively, actuation of thevehicle brake system in combination with a change in vehicle headinggreater than a threshold angle of change could trigger a change insensor viewing angle. In particular, the compass sensor information canbe compared to a threshold change of direction to determine if thecontrolled vehicle 100 is turning, and responsive thereto themicrocontroller 1105 can increase the sensitivity of the image sensor201 and associated circuitry to light sources in the direction that thevehicle is turning. In order to implement such a turning control, thecompass must generate vehicle heading information that themicrocontroller can use. Commercially available compasses of the typeused in vehicles generate an output which will operate within 2° to 3°of accuracy as the vehicle turns. As a consequence, the rate at whichthe vehicle heading is changing and the direction that the vehicle isturning can be determined from the compass output or GPS information. Inone embodiment, a compass is integrated into an electrochromic mirrorwith an automatic headlamp dimmer and the compass sensor signal isutilized to enhance the performance of the automatic headlamp dimmer.

If it is determined in step 2303 that a shift of the sensitivity patternis required, the image sensor will shift its sensitivity as indicated instep 2305. For example, the high sensitivity region 1 of the imagesensor will generally be in the center of the array such that lightsstraightforward of the vehicle will have the strongest impact on thelight control process. This is particularly advantageous as thebrightest light produced by the controlled vehicle illuminates straightout in front of the controlled vehicle. As the controlled vehicle turns,the center axis of each of the regions 1 through 5 will shift asindicated in step 2305. As a consequence, these regions shift in thedirection that the vehicle is turning such that they are centered to theright or left instead of being centered on the Y axis (FIG. 22). Thesensitivity of the pixels in array 301 will thus shift right as thevehicle turns right, and shift left as the vehicle turns left. Thedegree that the sensitivity field shifts can vary depending upon therate of change of the vehicle as well as the speed of the vehicle.

It is envisioned that the sensitivity can be changed using the lookuptable. In particular, the lookup table can contain respectiveintegration periods and thresholds according to the location of pixelssuch that the sensitivity is as shown in FIG. 22 when the vehicle istraveling straight. As the vehicle turns, the addresses associated withcolumns of pixels may be altered such that the integration periods andthresholds for pixels in the column will shift left or right.

If the change is not sufficient to change the sensitivity, as determinedin step 2303, the microcontroller will determine in step 2307 if thevehicle is going straight. The microcontroller will continue to monitorthe rate of change of the vehicle heading until the vehicle is headinggenerally straight, as indicated in step 2307. It will be recognizedthat a subroutine may be run once as an interrupt routine such thatother subroutines may be run in between execution of routine 2300.

It is further envisioned that adjustments may be made in the verticaldirection, in addition to the horizontal direction, for example, if achange in vehicle inclination is detected.

Light List History

Information about lights from previous frames can be useful inevaluating current frames. However, there is typically insufficientmemory in low cost microcontrollers to store previous frames so as toretain a complete history of each frame's content. To minimize thememory requirements for implementing the system, while retaining someuseful historical information, the brightness and location of one ormore of the brightest lights detected in one or more previous frames arestored for use when processing a later frame. The entire light list froma previous frame need not be stored, unless significant memory isavailable. It is also useful to store the brightness and location ofonly the brightest lights detected in one or more preceding frames. Thisinformation can be used to provide fast return to bright. After anoncoming car 105 has passed the controlled vehicle 100, it is useful toreturn to the high beam state as soon as possible. The night vision ofthe driver in the controlled vehicle may be temporarily impaired by thelights of the oncoming vehicle. The impact of this loss of night visionmay be minimized by providing as much scene illumination as possible, assoon as possible, following passage of a vehicle. The microcontroller1105 uses the light list history information to implement a fastreturn-to-bright. In particular, after step 2007 in FIG. 20, themicrocontroller 1105 can determine whether the current frame is suddenlyclear of bright light sources following a preceding frame that containeda very bright headlamp. In such a case, it is likely that the brightheadlamp has just passed the controlled vehicle. If this situationoccurs, the normal gradual, delayed fade-in period can be bypassed, orpartially bypassed, and the high beams can be incremented by more thanone state, such as by eight states (FIG. 19), to return to bright highbeams more quickly.

Another scenario where the opposite result is desired occurs when thecontrolled vehicle comes up behind a preceding vehicle 110. As thecontrolled vehicle 100 approaches the preceding vehicle 110, the imagesensor will detect the preceding vehicle's tail lights and dim thecontrolled vehicle's high beams responsive thereto. When the controlledvehicle moves to the side to drive around the slower preceding vehicle,the tail lights of the preceding vehicle will move out of the field ofview of the image sensor. However, if the controlled vehicle's brightlights are activated, they will shine into the eyes of the driver of thevehicle being passed via the exterior rearview mirror. This isparticularly problematic when the controlled vehicle is passing a truck,as it may take a long time to pass the truck. In this situation, themicrocontroller can include a decision step following step 2007 todetermine whether the previous frame included a tail light, and if so,to set a predetermined delay before the brights can be activated. Wherethe bright lights are dimmed responsive to preceding tail lights, a longdelay will thus be introduced before turning the high beams back ON. Forexample, the high beams may come on several seconds after the taillights move out of the scene being imaged.

The light history can also be used to select an integration period forthe image array sensor pixels. In particular, it is envisioned that theamplifier gain 303 for the pixels can have different gains or differentintegration intervals to increase the dynamic range of the light sensor.For example, three different gains or integration periods could besupported. A bright light will saturate all but the lowest gain, whereasa dim light such as a tail light can not be detected at a low gain. Thelight history can be used to remember that the sensor was washed outeven at low gain, and an ultra low sensitivity can be used to detectlights.

Another use for the light list history is determining traffic density.In particular, the traffic density can be ascertained from the timeperiod count between detecting oncoming headlights. Thus, in or near acity where traffic is heavier, the system can respond to that conditionby having a relatively long delay period. On the other hand, wheretraffic is light, such that oncoming traffic is less likely, the delayto turn on the bright lights can be short. It is envisioned that as thetraffic increases, the return-to-bright time period will lengthen,whereas as the traffic decreases, the return-to-bright period willshorten. It is further envisioned that a number of different criteriacould be used, such as the number of frames since a vehicle waspreviously detected or the percentage of time that the bright lightswere on over a predetermined sampling period. In these ways, the numberof objects detected over time can be used in the control of theheadlamps, and in particular, to at least partially inhibit turning onthe high beams.

It is envisioned that where a vehicle includes an electrochromic mirrorglare sensor to detect light from the rear of the vehicle, or any otherdevice having a rearward directed optical sensor, such as a rear visionsystem, additional information can be accessed which is useful forcontrolling the return-to-bright interval. In particular, when thebright headlights are dimmed because of tail lights from a precedingvehicle, the headlights can return to bright a predetermined time afterthe tail lights disappear from in front of the vehicle or whenheadlights are detected by the rear glare sensor in the rearview mirror,whichever occurs first. Where headlights from a trailing vehicle aredetected immediately prior to the disappearance of tail lights from apreceding vehicle, the use of the rearward sensor for detecting areturn-to-bright condition will be precluded. Additional considerationscan be used in making the return-to-bright decision. For example, aminimum and a maximum interval can be required before return-to-bright.

It is envisioned that the system will only provide a variablereturn-to-bright interval under certain conditions, such that thereturn-to-bright interval will typically be a default time interval.However, a fast return-to-bright interval will occur following acondition where really bright headlights are detected and thendisappear, as such bright lights will reduce the driver's night vision.Additionally, a slow return to bright condition would be used followingdisappearance of a preceding tail light since the driver's vision willnot have been impaired and it is desirable to avoid shining the highbeams into the eyes of a vehicle being passed.

It is further envisioned that integration periods in the current framemay be adjusted based upon measurements made in a previous frame. Inparticular, an extremely short integration period can be used for theimage sensor 301 where the lowest sensitivity measurements in a previousframe resulted in saturation of the light sensor. To the other extreme,where the previous frame's most sensitive measurements did result indetection of tail lights, a very long integration interval can be usedfor the image sensor 301 to look for tail lights in the current frame.Thus, where three integration periods are typically used, two additionalintegration periods can be selectively used when the conditionsnecessitate either an extremely short or long integration interval.

Another use of the light list history is to distinguish signs andreflectors based on the movement of the objects in the image over time.Over a sequence of frames, reflectors and signs will typically move morerapidly toward a side of the image than will vehicles travelinggenerally in parallel with the controlled vehicle. This characteristiccan be used to distinguish stationary objects from moving vehicles.

Automatic AIM Calibration

Variations in the orientation of the image array sensor relative to thewindshield angle may result in variations in aim of the sensor, whichmay negatively impact the performance of the dimmer. These variationscan be calibrated out over time using a maximum bound placed on theexpected variation in mounting. In general, on straight roads, distantoncoming headlamps will be coming from directly ahead of the opticalsensor system 102. The calibration system uses faint headlamps detectednear the center of the image, and preferably only those within a centerwindow corresponding to the expected mounting variations. Such headlampsmeeting certain criteria will have their position averaged with otherfaint headlamps meeting the same criteria. Over time, this average ofthese lights should correspond to the center of the field of view. Thisaverage value can be used to offset the image window relative to the X-and Y-axis in FIG. 22.

More particularly, in order to detect a flat straight road, from which amisalignment of the optical sensor system can be detected, a variety ofdifferent orientation inputs can be used. The speed of the vehicle maybe required to remain in a certain range between 35 to 50 mph. Thevehicle can be determined to be traveling straight using the compass,GPS, or monitoring the operation of the vehicle steering system. If theheading changes during the test, the measurement will be considered tobe in error. If the vehicle has equipment for providing an elevationmeasurement, any changes in the vehicle's elevation during thecalibration process will result in an error. The elevation measurementmay thus be used in determining whether to adjust the field of view.

A distant vehicle is initially detected by sensing white light near thecenter of the image, which is faint at the highest sensitivity (longestintegration period) of the image sensor. As this light gets brighter,the system monitors the orientation inputs to detect whether the roadcontinues to be flat and straight. If it remains flat and straight for aperiod of time which is at least twice the time period required for thevehicles 100, 105 to pass, the measurement will be valid. The centerpoint detected initially will then be averaged with previous validmeasurements, and the average measurement will be considered to be thecenter of the image. This will be the average X and Y coordinates, whichtogether will mark the center of the image sensor. This location can besaved in EEPROM or flash ROM.

Another method of aiming calibration is to take a very high gain imageand look for the reflection of the road. The average point where thisreflection occurs can be used to calibrate the aim.

Liquid Crystal Filter

An alternative optical system may include a liquid crystal filter 2405that can be used to selectively provide both a red and a blue filter,whereby red and blue images may be viewed by an image sensor 2401through a single lens structure 2403. In such a structure, the imagesensor 2401 need only have one imaging area (e.g., array area 702instead of array areas 702 and 703 as required with two lenses). Thefilter 2405 is implemented using a liquid crystal colored light switch2503 electrically connected to microcontroller 1105 through conductors2413 and 2411. The filter includes a neutral polarizer 2501, a liquidcrystal shutter 2503, a red polarizer 2505, and a blue polarizer 2507.The neutral polarizer 2501 and red polarizer 2505 are oriented withtheir polarizing axis aligned in one direction and the blue polarizer2507 is oriented with its polarizing axis oriented orthogonally to thered and neutral polarizer. The liquid crystal shutter 2503 isimplemented using a twisted neumatic (TN) liquid crystal shutterselectively energized under the control of microcontroller 1105. Inparticular, when the shutter is not energized, the liquid crystal devicetransmits the red light. When the liquid crystal is energized, theliquid crystal device transmits blue light.

It is thus possible to measure the relative intensities of two colors oflight using a single photo sensor or image area. In the unenergizedstate, all visible light is polarized in the horizontal direction by theneutral polarizer, rotated 90 degrees by the unenergized TN liquidcrystal cell to the vertical direction, and then all but the red lightwill be absorbed by the horizontal red polarizer. The red light willthen pass through the vertical blue polarizer. In the energized state,all visible light will be polarized in the horizontal direction by thehorizontal neutral polarizer, and not rotated by the energized TN liquidcrystal cell, all visible light will be transmitted by the horizontalred polarizer and all but blue light will be absorbed by the verticalblue polarizer. The liquid crystal device can be used as a high-speedlight switch to alternate between transmission of blue light and redlight. The relative intensities of the red and blue light components ofan object or light source can then be determined. Alternatively, a greenpolarizer can be substituted for either the red or blue polarizer toswitch between transmission of blue and green or red and green lightrespectively. Furthermore, a clear polarizer can be substituted for theblue polarizer to switch between red and clear.

Windshield Wiper

To improve dimmer performance when it is raining, it is useful tosynchronize the acquisition of images with the windshield wipers. Forexample, a signal can be provided from the wiper motors to indicate theposition of the wipers. Immediately after the wiper passes over thesensor, an image can be taken to look for cars. Most importantly, it isnecessary to avoid taking images while the wiper is over the imagesensor 301.

Where the controlled vehicle 100 includes a moisture sensor, themoisture sensor can monitor the windshield wiper. A moisture sensorproviding such information is disclosed in U.S. Pat. No. 5,923,027,entitled “MOISTURE SENSOR AND WINDSHIELD FOG DETECTOR,” issued to JosephS. Stam et al. on Jul. 13, 1999, the disclosure of which is incorporatedherein by reference thereto.

Deceleration

In addition to varying the image sensor operation depending upon thevehicle speed, other speed criteria can be used to control the operationof the vehicle headlamps. Turning on the high beams may be inhibitedwhen the vehicle is decelerating, when the brakes are actuated, or whenthe vehicle is traveling slowly. This prevents high beams from coming onwhen coasting to a stop or approaching an intersection. Deceleration canbe detected from the speed input to the microcontroller 1105 (FIG. 11).

Bad Pixel Calibration

An image sensor may contain one or more bad pixels. These bad pixels maymanifest themselves as extremely sensitive pixels which may cause“white-spots” in the image. Such “white-spots” will cause false lightdetection in the image if the sensor is not calibrated to remove themfrom calculations. During production tests, the location of these whitespots can be measured and stored in a programmable memory associatedwith microcontroller 1105. To compensate for such bad pixels duringnormal operation of the image array sensor, after an image is acquired,the pixel value at the white-spot location may be replaced with theaverage value of its neighboring pixels.

It is possible that a white spot may form during use of the imagesensor, such that it is not detected during production tests. Such asituation will cause the device to be inoperable. To avoid this problem,it is desirable to calibrate the white-spot out of the image afterrecognizing the bad pixel. In order to detect the white spot, it isnecessary to detect that the pixel remains “lit-up” in several images,and preferably over an extended period of time. A bad pixel will standout if it is repeatedly lit up when neighboring pixels are dark. Whensuch a pixel is detected, it can be added to the list of bad pixels.

It is envisioned that bad pixels can be periodically tested to determineif their performance has improved. This can be accomplished bymonitoring a sequence of dark images to determine whether the centerpixel is dark while the adjoining pixels are not dark.

Picket Fence

In controlling the vehicle headlamps, it is desirable to avoid acondition where the headlamp high beams flash ON and OFF at a relativelyrapid rate, which is particularly important if non-variable two-stateheadlamps are used. For example, a sign along the road can causeflashing of the headlights between bright and normal levels. This occurswhen reflections of the bright high beams from a sign are bright enoughto cause dimming of the high beams and reflections of headlight lowbeams are low enough that the bright high beams are turned ON. Thecondition can be avoided by having the system look at the object thatcaused the high beams to turn OFF. When this condition occurs, the lightlevel at that position is ignored while the pixels around the object arenot ignored for a predetermined number of cycles, such as ten cycles.The length of time that the object is ignored can be variable as afunction of the vehicle's speed. The higher the vehicle's speed, theshorter the period that the object will be ignored. During the timeperiod, the lights will be controlled using pixels other than thoseassociated with the object being ignored.

A reflector can be distinguished from an active source of light byflashing the vehicle headlights off. The time period that the headlightsare off is so short that the image sensor can sense the loss of lighteven though the human eye will not perceive, or barely perceive, thatthe lights were off. A light emitting diode headlamp describedhereinbelow can be turned OFF and ON very rapidly, such that it will beoff for such a short period of time. In operation, the microcontroller1105 controls the headlamp high or low beams to turn OFF, and controlsthe image sensor to image the scene during that brief time period thatthe headlamps are OFF. The OFF time period may, for example, be 10 ms.

Fog Detector

It is desirable for the vehicle to reliably detect a foggy condition,and in response thereto, to automatically turn ON or OFF front and rearfog lamps. Effective fog detectors have not been available heretofore.Fog may be detected by using the image sensor and optical system for theheadlight ON/OFF and headlight dimmer control. Fog can be detected by areduced scene contrast along with scene ambient light leveldeterminations. The ambient light can be determined from the mean greyscale value of the pixels imaging the forward scene or by a separateambient light sensor, such as the ambient light sensor used for theelectrochromic mirror. It is envisioned that the mean can be a clippedmean value. The variance of the grey scale values of the pixels providesa measure of the contrast in the image. Alternatively, the variance canbe determined from the standard deviation of the pixels and, inparticular, when the standard deviation is less than a standarddeviation threshold level, the presence of fog is identified responsiveto which the fog lights can be turned ON. Alternatively, the individualdifferences between the average pixel level and each individual pixellevel can be added for the entire image sensor, and when this totalvariance exceeds a variance threshold level, the presence of fog isdetected. Either of these examples of fog criteria can be used as ameasure of contrast.

Several additional factors may also be considered. The contrast valuemay be an average contrast value over several images. The contrast maybe determined in row-wise fashion to determine the level of fog. Variousregions of the scene may be considered independently. If two colorlenses are present as in the headlamp dimmer, color information may beused. The actual values of the brightness/contrast ratios and the properimage sensor exposure times should be determined experimentally. Thedevice can be set to only operate between a predetermined range ofbrightness levels, such that if the ambient level is too high or toolow, the fog detector will not operate, and manual override will berequired. The ambient light conditions for fog may, for example, bebetween 1 and 1000 lux. Both the image sensor and the ambient lightsensor can be used to detect fog. If the ambient light level detected bythe ambient light sensor is within the appropriate range, an image ofthe forward scene is acquired with a sensitivity set to the average greyscale value of the pixels (e.g., 128 lux). If the contrast at a givenbrightness level is below a predetermined threshold level, it isdetermined that fog is present.

Led Headlamps for Headlamp Steering and Headlamp Flashing

It has long been considered to be desirable to provide headlamps thatcan be steered in the direction that the vehicle is turning. It is alsodesirable to provide forward lighting that can be turned OFF, orsubstantially attenuated, for such a short period of time that thedriver does not notice that the lights are OFF. In this way, reflectionsdo not exist during acquisition. Although a light emitting diode (LED)lamp can be used to provide these features in a cost-effective manner,LED lamps producing enough light to implement a vehicle headlamp are notcommercially available. LEDs suffer from a number of disadvantages thatlimit their application in vehicle headlamps, not the least of which arethe relatively small amounts of light produced by LEDs andmanufacturability limitations when incorporating LEDS having exoticconstructions. Because of these disadvantages, LED lamps have not beenused to implement a vehicle headlamp despite the fact that LEDs are morerugged, more energy efficient, and significantly longer lasting thanother light technologies. Additionally, means of producing white lightfrom LEDS have only recently become practical as discussed in U.S. Pat.No. 5,803,579, entitled “ILLUMINATOR ASSEMBLY INCORPORATING LIGHTEMITTING DIODES,” issued to John K. Roberts et al. on Sep. 8, 1998, thedisclosure of which is incorporated herein by reference.

An LED headlamp 2600 is disclosed in FIGS. 26 and 26 b. The LED headlampcan be used to very briefly flash OFF, or dim, the headlamps during animage sampling interval. The LED headlamp 2600 includes a heatextraction member 2601 that serves as a support for mountingsemiconductor optical radiation emitters 2603, 2605. Where thesemiconductor optical radiation emitters 2603, 2605 are electricallyconnected to the heat extraction member, the heat extraction memberprovides an electrical connection to the semiconductor optical radiationemitters in addition to providing a thermal path for removing heatgenerated within the semiconductor optical radiation emitters duringoperation. It is envisioned that the emitters 2603, 2605 can beelectrically isolated from the heat extraction member such that the heatextraction member only provides a thermal path. Each of the emitters2603 is connected to electrical conductor strip 2607 through a wire bond2609 and a resistor 2611. Each of the emitters 2605 is connected toelectrical conductor strip 2613 through a bonding wire 2615 and aresistor 2617.

The heat extraction member 2601 may be constructed of any suitablematerial, and may be formed in any desired configuration. The front faceof the illustrated heat extraction member is generally rectangular inshape, including 33 wells, each of which receives semiconductor opticalradiation emitters 2603, 2605. The back of the heat extraction memberincludes fins 2621 that provide a large surface for thermal dissipationto the ambient air. It is envisioned that the heat extraction member mayalternatively have other configurations which enable light steering,such as being generally convex, shaped like a portion of a cylinder sidewall, an elongate bar to extend across the front of the vehicle, or aplurality of joined planar surfaces extending at different angles, orthe like. The heat extraction member can be chamfered, or otherwisecontain extensions, slots, holes, grooves and the like, and mayincorporate depressions such as collimating cup or other form to enhanceoptical performance. The illustrated heat extraction member includeselliptical cups 2602. The heat extraction member may be composed ofcopper, copper alloys such as beryllium, aluminum, aluminum alloys,steel, or other metal, or alternatively of another high thermalconductivity material such as ceramic. Preferably, the heat extractionmember is constructed from an electrically and thermally conductivemetal. Such materials are commercially available from a wide variety ofsources.

The cups 2602 are formed in the top surface of the heat extractionmember for receipt of the semiconductor optical radiation emitters. Theillustrated cups 2602 are elliptical to accommodate two emitter chips2603, 2605 side by side. However, the cups may be of any suitable shapesuch as round, elliptical, rectangular, square, pentagonal, octagonal,hexagonal, or the like. The elliptical cups have the advantage ofaccommodating more than one emitter while providing an efficientreflector for projecting light outwardly in a desired radiation pattern.It is envisioned that the region of the heat extraction member directlyunderlying the point of attachment of the semiconductor opticalradiation emitter may be coated with nickel, palladium, gold, silver orother material including alloys, in order to enhance the quality andreliability of the die attach. Other thin-layered materials may beoptionally inserted between the emitter and the heat extraction memberto achieve a variety of desired effects without departing from the scopeand spirit of the present invention. The material preferably provides anelectrical connection between the emitters and the heat extractionmember whereby the heat extraction member can provide a referencepotential, which, for example, may be ground potential. The materialsare preferably adhesive, electrically insulative, and either conductiveor patterned composite of electrically insulative and conductivematerials, and without significantly impeding thermal transfer, may beused to support, bond, electrically connect or otherwise mount to theemitter to the heat extraction member. The region of the heat extractionmember within optical-enhancement cup feature may be coated with silver,aluminum, gold or other suitable material to increase reflectance andimprove the optical efficiency of the device. The area outside of theencapsulant may be coated with nichrome, black oxide or other highemissivity treatment to improve radiative cooling.

The connection of the semiconductor optical radiation emitter ispreferably by the use of a special type of electrically conductiveadhesive die-attach epoxy. These adhesives normally achieve goodelectrical conductivity by inclusion metallic fillers such as silver inthe adhesive. Suitable die-attach adhesives are well known in the artand may be obtained from Quantum Materials of San Diego, Calif. fromAblestik division of National Starch and Chemical, and EpoTek ofBillerica, Mass. Alternatively, solder may be used as a means ofattaching the LED chip to the heat extraction member in someembodiments. Whether attached by electrically conductive adhesive orsolder, the bond establishes good electrical and thermal conductivitybetween the emitter and the heat extraction member. In the case wherethe emitters having electrodes manifest as conductive bond pads at thetop of the LED chips rather than at their base, the electricalattachment of all of the electrodes is by wire bond rather than bydie-attach to the heat extraction member.

The semiconductor optical radiation emitters comprise any component ormaterial that emits electromagnetic radiation having a wavelengthbetween 100 nm and 2000 nm by the physical mechanism ofelectroluminescence, upon passage of electrical current through thematerial or component. For purposes of generating head lightillumination, different emitters can be used in the wells to generatethe light, such as: all amber emitters placed in each of the wells;amber and cyan emitters positioned in each of the wells; red-orange andcyan emitters placed in each of the wells; cyan and amber emittersplaced in some of the wells and red-orange and cyan emitters in otherwells; phosphorous emitters in each of the wells; or the like. It isenvisioned that four out of the five wells can have blue-green and amberemitters and one in five wells can have red-orange and amber emitters.Such an arrangement will produce a white light for illuminating the pathof the vehicle.

The semiconductor optical emitter may comprise a light emitting diode(LED) chip or die as are well known in the art, light emitting polymers(LEPs), polymer light emitting diodes (PLEDs), organic light emittingdiodes (OLEDs), or the like. Such materials and optoelectronicstructures made from them are electrically similar to traditionalinorganic materials known to those skilled in the art, and are availablefrom a variety of different sources. Semiconductor optical radiationemitter, or emitter, as used herein refers to each of these and theirequivalents. Examples of emitters suitable for headlamps include AlGaAs,AlInGaP, GaAs, GaP, InGaN, SiC, and may include emissions enhanced viathe physical mechanism of fluorescence by the use of an organic orinorganic die or phosphor. LED chips suitable for use in the presentinvention are made by companies such as Hewlett-Packard, NichiaChemical, Siemans Optoelectronics, Sharp, Stanley, Toshibe, Lite-On,Cree Research, Toyoda Gosei, Showa Denko, Tyntec, and others. Such chipsare typically fashioned approximately in square base between 0.008″ and0.016″ long on each side, a height of about 0.008″ to 0.020″. Toimplement the headlight, a larger chip having a square base larger than0.020″ may be used, and in particular it is advantageous to have a sizegreater than 0.025″ to 0.035″ to generate light. An array of suchemitters mounted to a substantial heat sink permits an LED lamp togenerate sufficient light to operate as a vehicle headlight. Details ofan emitter that can be used can be found in U.S. patent application Ser.No. 09/426,795, entitled “SEMICONDUCTOR RADIATION EMITTER PACKAGE,”filed on Oct. 22, 1999, by John K. Roberts et al., now U.S. Pat. No.6,335,548, the disclosure of which is incorporated herein by reference.An electrical path for supplying control signals to the emitters isprovided through conductors. The conductors are electrical stripsapplied to a surface of an insulative layer, the insulative layer beingmounted to the top surface of the heat sink. The insulative layer may bea circuit board including openings over the cups, or it may comprise anepoxy or plastic layer. The conductor may be any suitable electricallyconductive material such as copper, aluminum, an alloy, or the like, andmay advantageously comprise circuit traces applied to the insulatingmaterial by conventional means. The circuit traces disclosed in FIG. 26a illustrate a pattern that may be used where separate supply isprovided for the different emitters.

An encapsulant is a material or combination of materials that servesprimarily to cover and protect the semiconductor optical radiationemitter and wire bonds. The encapsulant is transparent to wavelengths ofradiation. For purposes of the present invention, a substantiallytransparent encapsulant refers to a material that, in a flat thicknessof 0.5 mm, exhibits greater than 20% total transmittance of light at anywavelength in the visible light range between 380 nm and 800 nm. Theencapsulant material typically includes a clear epoxy or other thermosetmaterial, silicone, or acrylate. Alternatively, the encapsulant mayconceivably include glass or thermoplastic such as acrylic,polycarbonate, COC, or the like. The encapsulant may include materialsthat are solid, liquid, or gel at room temperature. The encapsulant mayinclude transfer molding compounds such as NT 300H, available form NittoDenko, or potting, encapsulation or other materials which start as asingle part or multiple parts and are processed with a high temperaturecure, two part cure, ultra-violet cure, microwave cure, or the like.Suitable clear encapsulants may be obtained from Epoxy Technology ofBillerica, Mass. from Nitto Denko America, Inc., of Fremont, Calif. orfrom Dexter Electronic Materials of Industry, Calif.

The encapsulant may provide partial optical collimation or other beamformation of electromagnetic energy emitted by the emitter and orreflected by the surface of heat extraction member. The encapsulant alsoserves as a chemical barrier, sealant, and physical shroud providingprotection of emitters, internal adhesives such as bonds, bond pads,conductor wires, wire bonds and internal surfaces of heat extractionmember and electrical leads from environmental damage due to oxygenexposure, exposure to humidity or other corrosive vapors, solventexposure, mechanical abrasion or trauma, and the like. The encapsulantprovides electrical insulation. The encapsulant may also provide forattaching or registering to adjacent components such as secondaryoptics, support members, secondary heat extractors, and the like.

The encapsulant may comprise a heterogeneous mass of more than onematerial, wherein each material occupies a portion of the overallencapsulant volume and provides a specialized function or property. Forexample, a stress relieving gel such as a silicone “glob top” may beplaced over the emitter and wire bonds. Such a localized stressrelieving gel remains soft and deformable and may serve to cushion theemitter and wire bonds from stress incurred during subsequent processingof the component or due to thermal expansion to shock. A hard moldingcompound such as an epoxy may then be formed over the stress relievinggel to provide structural integration for the various features of thecomponent, to retain the electrical leads, to protect the internalmechanisms of the component from environmental influences, toelectrically insulate the semiconductor radiation emitters, to providevarious optical moderation of radiant energy emitted by the emitter ifdesired. Additionally, the filler used within the stress relieving gelmay advantageously include a high thermal conductivity material such asdiamond powder. Diamond is a chemically inert substance with anextremely high thermal conductivity. The presence of such a material maysignificantly increase the thermal conductivity of the gel and providean additional path for heat generated in the emitter chip to reach theheat extraction member and ambient environment where it can bedissipated. Such additional heat extraction path will increase theefficiency of the emitter, and thus the light output of the lamp. Theencapsulant, and manufacture of a device using such an encapsulant, isdescribed in greater detail in U.S. Pat. No. 6,335,548, incorporated byreference hereinabove.

A steerable light emitting diode headlamp 2700 is illustrated in FIG.27. The headlamp includes a heat extraction member 2701 having two lampsections 2703 and 2705. The heat extraction member 2701 is configured toinclude the two lamp sections 2703 and 2705, each of which issubstantially identical to lamp 2601, presenting two front faces at anangle of 5° to 45°, and may advantageously be angled at approximately15°.

Another alternative design for a headlamp for providing low and highbeams is illustrated in FIG. 28. The headlamp 2800 includes a heatextraction member 2801 having two lamp sections 2803 and 2805. The heatextraction member 2801 is configured to include the two lamp sections2803 and 2805, each of which is substantially identical to lamp 2601.The heat extraction member provides two front faces at an angle of 1° to2°, and may advantageously be angled at approximately 1.5°. The angle isexaggerated in FIG. 28 so that the angled surfaces are readily visible.

By controlling the selection of the LEDs that are illuminated, theheadlights can be aimed. It is also envisioned that the front of the carcan have LED chips positioned in a bar running across the front grill.

Whereas the above embodiment takes advantage of directly mounting anumber of chips on a common heat sink, it is also envisioned that anarray of discrete LED lamps can be used to implement an LED headlamp2650 (FIGS. 26 c and 26 d). Each discrete LED lamp preferably includes aheat extraction member for dissipating power generated by the emittersto obtain a brighter light level without damaging the LED components. Aparticularly advantageous high power LED lamp which is uniquely adaptedfor conventional manufacturing processes is disclosed in U.S. patentapplication Ser. No. 09/426,795, entitled “SEMICONDUCTOR RADIATIONEMITTER PACKAGE,” filed on Oct. 22, 1999, by John K. Roberts et al., nowU.S. Pat. No. 6,335,548, the disclosure of which is incorporated hereinby reference. Other LED lamps that could be used are commerciallyavailable from LED manufacturers such as Hewlett Packard Company.

The LED lamps 2650 (only some of which are numbered) are mounted to acircuit board 2652 and heat sink 2654 (FIG. 26 d). The circuit board canprovide a secondary heat sink, where the conductive layer 2660 of thecircuit board exposed to ambient air, by thermally coupling the heatextraction member 2664 of each LED lamp to the conductive layer 2662 ofthe circuit board. The heat extraction member of the LED lamp is alsothermally coupled to the heat sink 2654. In the illustrated embodimentof FIGS. 26 c and 26 d, the thermally conductive material 2670 ispositioned in a hole through the circuit board below the heat extractionmember. The thermally conductive material is thicker than the circuitboard, and resilient. For example, the thermally conductive material canbe provided using a preformed thermal coupler such as a silicon based,cut resistant material commercially available from Bergquist, andidentified as Silipad 600. Packages for LED lamps using the heatextraction member are disclosed in U.S. patent application Ser. No.09/425,792, entitled “INDICATORS AND ILLUMINATORS USING A SEMICONDUCTORRADIATION EMITTER,” filed on Oct. 22, 1999, by John K. Roberts et al.,now U.S. Pat. No. 6,335,548, the disclosure of which is incorporatedherein by reference thereto.

Where red-green-blue or binary complementary lighting is used, it isenvisioned that only selected chips will be flashed OFF when attemptingto distinguish reflective objects from lamps. Thus, for binarycomplementary emitters, only amber emitters need to have a brieflyreduced intensity. Additionally, it will be recognized that instead ofturning the headlamps OFF, the light level can be reduced to a level atwhich reflections will be below the pixel threshold at which the imagesensor assembly will detect an object.

Surface Mounted Filter for Sensor

A method by which a filter can be directly deposited onto asemiconductor light sensor 201 will now be described with respect toFIGS. 29 a through 29 d. In the first step, a photoresist 2877 isdeposited over the entire wafer 2972. The photoresist 2877 may be anysuitable commercially available photoresist material. Portions of thephotoresist may be removed such that the remaining photoresist ispatterned to cover only those areas on the surface of the waferrequiring protection from the optical coating deposition, such as thebonding pad 2975, as shown in FIG. 29 b. The optical film coating 2979is then applied to the surface of the die 2972 as shown in FIG. 29 c.

The thin film 2979 is deposited directly on the light sensor 2932 inmultiple layers. The red and cyan filters, if red and cyan filters aredesired, will be applied separately. An example of a cyan filter will benow be described. To make a cyan filter, the layers of titanium dioxide(TiO₂) and silicon dioxide (SiO₂) described in Table 1 can be used. Tomake a red filter, the layers described in Table 2 can be used. Thelayer number is the order in which the material is applied to the wafersurface.

TABLE 1 Layer Material Thickness(nm) 1 SiO₂ 170 2 TiO₂ 124 3 SiO₂ 10 4TiO₂ 134 5 SiO₂ 160 6 TiO₂ 79 7 SiO₂ 164 8 TiO₂ 29 9 SiO₂ 168 10 TiO₂ 6811 SiO₂ 164 12 TiO₂ 33 13 SiO₂ 163 14 TiO₂ 69 15 SiO₂ 154 16 TiO₂ 188 17SiO₂ 148 18 TiO₂ 88 19 SiO₂ 319

TABLE 2 Layer Material Thickness(nm) 1 TiO₂ 68 2 SiO₂ 64 3 TiO₂ 35 4SiO₂ 138 5 TiO₂ 57 6 SiO₂ 86 7 TiO₂ 50 8 SiO₂ 78 9 TiO₂ 73 10 SiO₂ 94 11TiO₂ 54 12 SiO₂ 89 13 TiO₂ 52 14 SiO₂ 87 15 TiO₂ 50 16 SiO₂ 74 17 TiO₂28 18 SiO₂ 61 19 TiO₂ 49 20 SiO₂ 83 21 TiO₂ 48 22 SiO₂ 78 23 TiO₂ 48 24SiO₂ 91After of the layer are deposited, the photoresist is lifted off using aconventional lift off process, leaving the film deposited over the lightsensitive region, but not over the bonding pads, as shown in FIG. 29 d.The resulting die can be encapsulated to provide the image array sensorin conventional packaging.

The characteristics of the filter produced according to Table 1 andTable 2 are illustrated in FIGS. 30 and 31. In particular, the redfilter will attenuate light below 625 nm whereas the blue filter willpass light between approximately 400 nm and 625 nm. Both filters willpass light above 800 nm. An infrared filter can be utilized to reducethe effect of infrared light on the performance of the headlight dimmer.

Those skilled in the art will recognize that the filters describedherein are exemplary, and that other filters, material thickness couldbe used to implement the filter function. Other materials could beapplied in a similar manner to provide these or other filtercharacteristics.

Those skilled in the art will recognize that the layer thicknesses arerounded to the nearest nanometer. Although there will be some tolerancepermitted, good precision in the stack construction is required. It willalso be recognized that the layer thicknesses are exemplary. By surfacemounting the filters, the cost of providing the image sensor can begreatly reduced, as components and manufacturing complexity are reduced.Additionally, an infrared filter can be applied as a coating between thepixels and the red and blue filters.

Package

An alternate optical sensor assembly 3200 is disclosed in FIG. 32.Optical sensor assembly 3200 includes a base substrate 3202, which istransparent. The substrate may be manufactured of any suitable materialsuch as a transparent polymer, glass, or the like. Alternately, it isenvisioned that the base substrate may be manufactured of a commerciallyavailable infrared interference filter such as those describedhereinabove. Alternatively, a thin film filter may be attached to atransparent glass element to make the base substrate 3202.

The lower surface of the base substrate has conductive strips 3210 forconnection to the image array sensor 3212. The base substrate ispreferably an electrical insulator, whereby the strips can be anysuitable electrically conductive material applied directly to the lowersurface of the base substrate by conventional manufacturing processes.Alternatively, if the base substrate is an electrical conductor, thestrips can be applied to an electrical insulator, which is in turnapplied to the base substrate.

The image array sensor 3212 is flip chip bonded to the lower surface ofthe base substrate 3202 by soldering pads 3211. A dielectric material3214, such as an epoxy, encloses the image array sensor. The dielectricmaterial preferably bonds with the conductive strips 3210 and thetransparent element 3202. Clips 3204 and 3206 clip onto the edges of thebase substrate and make electrical contact with respective conductivestrips 3210. A respective clip can be provided for each of theconductive strips 3210. Leads 3213, 3215 extend from the clips forinsertion into support substrate 3201, which can be a printed circuitboard, a housing, or the like. The support substrate is preferably aprinted circuit board carried in a housing, such as a rearview mirrormount housing. The stops 3205, 3207 limit the length of the leads thatcan be inserted into the support substrate. Alternatively, the clips canbe configured for surface mounting. Examples of surface mountable clipsinclude NAS Interplex Edge Clips, from NAS Interplex, an InterplexIndustries Company located in Flushing, N.Y. USA.

Packages for mounting the image array sensor are described in U.S. Pat.No. 6,130,448 entitled “OPTICAL SENSOR PACKAGE AND METHOD OF MAKING THESAME,” filed on Aug. 21, 1998, by Frederick T. Bauer et al., thedisclosure of which is incorporated herein by reference.

The assembly of the lens to the base substrate can be provided in thesame manner as described hereinabove. In particular, the lens structureis carried on the base substrate 3202. The lens structure can beidentical to the lens structure 202, and thus can be half clear and halfred, or half cyan and half red, or entirely clear where a color filteris applied directly to the surface of the image sensor as describedhereinabove with respect to FIGS. 29 a-29 d. To make the lens assembly,a transparent member 3222 and a UV curable adhesive 3222 can be used.The transparent member can be an epoxy member like members 230, 802. TheUV curable adhesive can be identical to adhesive 232.

Radar

A wave transceiver 101 (FIG. 1) can be used to acquire additionalintelligence usable for controlling the operation of the controlledvehicle 100. The wave transceiver can be used without the image arraysensor 102, or it can be used with the image array sensor. The wavetransceiver device 101 is mounted on a vehicle 100 and oriented in agenerally forward direction. The wave transceiver device 101 ispositioned to receive the reflections of waves emitted by thetransceiver device after they have reflected off of objects in front ofthe vehicle 100. The wave-emitting device may be a radar systemoperating, for example, in a frequency range exceeding 1 GHz (77 GHz isdesignated for vehicular radar in many European countries) or an opticalradar utilizing, for example, laser diodes for the wave-emitting device.Alternatively, the wave-emitting device 101 may be an ultrasonictransducer emitting ultrasonic waves. The wave-emitting device may bescanned across the forward field to cover various angles. For thepurposes of this invention, the term “radar” will be used to encompassall of these concepts. The wave transceiver should not be interpreted asbeing limited to any specific type or configuration of wave transmittingor wave receiving device. The transmitter and receiver may each bemounted within a different respective housing or they both may bemounted in a common housing.

A radar processing system 3300 (FIG. 33) controls the wave transmittingsection 3301, 3302 and interprets the signal received by the wavereceiving section 3304, 3305 to determine the presence of objects aswell as the speed and direction of such objects. A headlamp controller3303 receives target information from the radar processing system, andmay optionally also receive signals from the vehicle speed sensor (suchas a speedometer) and a vehicle direction sensor (such as a compass) andgenerates a control signal which determines the state of the vehicleheadlamps 111. The communication between the radar processing system,the vehicle speed sensor, the vehicle direction system, and the vehicleheadlamps may be by one of many mechanisms including direct wiringthrough a wiring harness or by a vehicle communication bus such as theCAN bus. Additionally, systems such as the radar processing system andthe headlamp controller may be implemented by a single integratedprocessor, multiple processors, digital signal processors,microcontrollers, microprocessors, programmable logic units, orcombinations thereof.

More particularly, the system contains a radar system which includes awave transmitting section and a wave receiving section. The wavetransceiver 101 includes an emitter 3301 and a receiver 3304. Thetransmitter may be implemented using an antenna for a conventionalradar, a light source for optical radar, an ultrasonic emitter, anantenna system for a Doppler radar system, or the like. The receiver3304 may be implemented using an antenna in a conventional radar system,a light receiving element in an optical radar system, an ultrasonicreceiver, a wave guide antenna in a Doppler radar system, or the like. Adriver 3302 is connected to the emitter 3301 to condition signals fromcontroller 3303 so that emitter 3301 produces signals, which, whenreflected, can be detected by the receiver 3304. The driver 3302 may beimplemented using a pulse modulator, a pulse shaper, or the like. Thereceiver 3304 is connected to a conditioning circuit 3305 whichconditions the signals detected by the receiver for further processingby controller 3303. The conditioning circuit may include a demodulator,a filter, an amplifier, an analog-to-digital converter (ADC),combinations thereof, and the like. The controller 3303 may beimplemented using a microprocessor, a digital signal processor, amicrocontroller, a programmable logic unit, combinations thereof, or thelike.

The operation of radars to determine the presence of objects relative toa vehicle are well known, and will not be described in greater detailhereinbelow. For example, the time between the transmitted wave and thedetection of the reflected wave may be used to determine the distance ofan object. The movement of an object over successivetransmission/reception cycles may be used to determine an object'srelative speed and direction. Doppler radar may also be used todetermine the objects speed. The magnitude of the reflected wave may beused to determine the size or density of the detected object. Operationof radars is well known, and will not be described in greater detailherein.

In order to properly control the high beam state of the controlledvehicle 100, it is necessary to determine if an object detected by theradar is a vehicle or a stationary object and, if a vehicle, whether thevehicle is an oncoming vehicle 105 or preceding vehicle 110. This can beaccomplished by comparing the speed and direction of the object with thespeed and direction of the controlled vehicle 100. The speed anddirection of travel of the object is obtained using the radar principlesdescribed above. The speed of the control vehicle 100 may be obtainedfrom a speed sensor on the vehicle, a global positioning system (GPS)system, or the like. The direction of the controlled vehicle 100 may beobtained from a compass sensor, a steering wheel turn indicator, a GPS,or the like.

Once this information is obtained, a simple set of criteria is appliedto determine if the object is a vehicle or a stationary object. If anobject is stationary, it will be moving in a direction opposite thecontrolled vehicle 100 at the same speed as the controlled vehicle. Ifan object is an oncoming vehicle 105, it will be traveling in adirection approximately opposite the controlled vehicle 100 at a speedsubstantially faster than that of the controlled vehicle. Finally, if anobject is not moving relative to the controlled vehicle or the object ismoving at a rate substantially slower than the controlled vehicle, theobject is likely a preceding vehicle 110. The distance at which the highbeams are dimmed may be a function of the speed of the controlledvehicle and may be a function of the angle between an axis straightforward of the controlled vehicle 100 and the oncoming vehicle 105 orpreceding vehicle 110.

In a more advanced system, the headlamp control system not only controlsthe high/low beam state of the headlamps 3311, 3312 and high beam lamps3314, 3315 based on the presence of one or more vehicles, but may varythe brightness of the high beam headlamps and low beam headlamps toprovide a continuous transition between the two beams as a function ofthe distance to the nearest other vehicle, thus maximizing the availableluminance provided to the driver of the controlled vehicle withoutdistraction to the other driver. A continuously variable headlamp systemis disclosed in U.S. Pat. No. 6,049,171 entitled “CONTINUOUSLY VARIABLEHEADLAMP CONTROL,” filed on Sep. 18, 1998, by Joseph S. Stam et al., thedisclosure of which is incorporated herein by reference thereto. Thesystem may also vary the aim of the controlled vehicle headlamps in thevertical direction. The system may be configured to transition betweenmore than two beams or may be configured to perform a combination ofaiming and varying the brightness of one or more lamps. The headlampprocessing system may also use the vehicle direction input to determinethe proper horizontal aim of the headlamps to provide betterillumination when traveling on curves. An LED headlamp that facilitatesaiming is disclosed above.

A light sensor 3320 can be used to detect ambient light levels and mayoptionally provide other light conditions. The light sensor may beimplemented using a non-imaging sensor such as a silicon photodiode, aparticularly advantageous photodiode disclosed in U.S. patentapplication Ser. No. 09/237,107, entitled “PHOTODIODE LIGHT SENSOR,”filed by Robert Nixon et al., now abandoned, the disclosure of which isincorporated herein by reference thereto, although other non-imagingphotocells could be used such as cadmium sulphide (CdS) cells, or thelike. The light sensor 3320 can alternately be implemented using anoptical image sensor 102 in addition to, or instead of, a non-imaginglight sensor.

The optical system may contain filters to determine the color of a lightsource. The combination of an optical system with a radar system maybetter overcome the limitations present if only one or two systems areused independently. For example, if a radar system is used independentlyof an imaging system, an oncoming or preceding vehicle waiting at anintersection would be perceived by the radar system as a stationaryobject. However, by combining an optical sensor with the radar system,the lights on the waiting vehicle would indicate that a vehicle ispresent and the radar can determine the actual distance to the vehicle.In general, if an optical system is used, the optical system may be usedto determine the presence of oncoming or preceding vehicles and theradar system may be used to determine the actual distance to suchoncoming or preceding vehicle as well as the speed of that vehicle. Inthis manner, the radar detector and the imaging sensor can be used toverify the presence of other objects. Additionally, the light sensorscan be used to determine light conditions, such as ambient light levels.

The presence of the radar system on the vehicle may enable otherfeatures other than headlamp control to be implemented utilizing thesame components as the headlamp radar, thus reducing the cost of the twosystems combined. Such system may include, for example, adaptive cruisecontrol, obstacle warning systems, collision avoidance systems,autonomous driving systems, or the like. In this case, the wavetransmitting section, wave receiving section, and radar processingsystems could be shared by all features while each feature has its ownprocessing system for determining a course of action based upon theinformation received from the radar processing system. It is alsopossible to integrate the processing systems from each feature into asingle processor.

A radar system is described in U.S. patent application Ser. No.09/531,211 entitled “AUTOMATIC HEADLAMP CONTROL SYSTEM,” filed on Mar.20, 2000, now U.S. Pat. No. 6,403,942, the disclosure of which isincorporated herein by reference.

While the invention has been described in detail herein in accordancewith certain embodiments thereof, many modifications and changes may beeffected by those skilled in the art without departing from the spiritof the invention. Accordingly, it is intended that the appended claimsnot be limited by way of details and instrumentalities describing theembodiments shown herein.

1. A vehicle control system, comprising: a first image sensor focused ata distance for detecting lights; a second image sensor focused near thewindow; windshield wipers; headlights; and a controller coupled to thefirst image sensor, the second image sensor focused near the window, thewindshield wipers, and the headlights, wherein the controller isoperable to control the headlights responsive to the first image sensor,control the windshield wipers to operate in response to the second imagesensor when rain is on the window, and the controller being furtheroperable to control the first image sensor to image a scene when thewindshield wipers are not in front of the first image sensor.
 2. Thevehicle control system of claim 1, wherein said first image sensor isdisposed in a rearview mirror assembly.
 3. The vehicle control system ofclaim 2, wherein said second image sensor is disposed in a rearviewmirror assembly.
 4. The vehicle control system of claim 3, wherein saidcontroller is disposed in a rearview mirror assembly.
 5. The vehiclecontrol system of claim 1, wherein said second image sensor is disposedin a rearview mirror assembly.
 6. The vehicle control system of claim 5,wherein said controller is disposed in a rearview mirror assembly. 7.The vehicle control system of claim 1, wherein said controller isdisposed in a rearview mirror assembly.
 8. The vehicle control system ofclaim 1, wherein said controller identifies and determines thebrightness of light sources in images obtained from said first imagingsensor and controls a beam pattern of headlamps of the vehicle as afunction of the brightness of light sources within the images.
 9. Amethod of making an optical sensor assembly package including an imagesensor die comprising the steps of: covering the die with an epoxy;positioning an adhesive on the epoxy; positioning an optical structureon the adhesive; curing the adhesive to form the integral package; andmolding a lens structure to include a lens.
 10. The method of claim 9,further including the step of molding a optical structure to include alens.
 11. The method of claim 10, wherein the step of covering includescovering the die with a glob-top epoxy.
 12. The method of claim 9,further including the step of molding an optical structure to include afirst lens having a filter for removing light of at least a first wavelength.
 13. The method of claim 12, wherein the step of molding anoptical structure further includes molding a second lens passing atleast light of the first wave length.
 14. The method of claim 13,wherein the step of covering includes covering the die with a glob-topepoxy.
 15. The method of claim 9, wherein the step of covering includescovering the die with a glob-top epoxy.
 16. The method of claim 15,further including the step of applying at least one filter to a surfaceof the image sensor die.
 17. The method of claim 9, further includingthe step of applying at least one filter to a surface of the imagesensor die.
 18. The method of claim 17, wherein the step of applying atleast one filter includes applying two or more filters to respectiveareas on the surface of the sensor.
 19. The method of claim 9, furtherincluding the step of mounting an electronically controlled filter.